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Chapter | 


Introduction to open world 
learning 


Research, innovation and the challenges 
of high-quality education 


Bart Rienties, Regine Hampel, Eileen Scanlon and 
Denise Whitelock 


I.I Introduction 


The main objective of this book Open World Learning: Research, Innovation and the 
Challenges of High-Quality Education is to establish an informed theoretical and 
methodological basis for research and practical application of open world learning. 
With the global pandemic, nearly every person on this planet has been touched by 
the impact of COVID-19. In part, the unprecedented spread of COVID-19 has 
been due to the growth of technology and the interconnected nature of our societ- 
les across this planet. Thanks to global travel afforded by technology, a "small" 
infection in one part of the world rapidly infected millions of people across the 
globe in less than four months. At the same time, thanks to innovative technology 
millions of teachers and students were able to continue to learn online when many 
societies went into lockdown. Similarly, the unprecedented global race to find a 
vaccine would not have been able to complete successfully so quickly without 
technology and the combined forces of scientific research. 

In these unprecedented and strange times, the power and limitations of technol- 
ogy have become even more visible to many. The rate of transition of learning in 
the 21st century has undergone both subtle and radical transformation as a result 
of COVID-19. Some of these changes were already starting to become visible in 
several parts of societies and educational systems in particular, such as moves 
towards blended and online education (Hampel, 2019). Other changes like work- 
ing from home, attending online concerts, or creating Zoom discos have led to 
substantial organisational and personal changes in our daily lives, which we may 
not have seen for a decade without COVID-19. 

Open world learning gives unprecedented access to information, knowledge, 
and education and provides support to learners across the globe. However, it is not 
the technologies themselves that represent the biggest change, but the opportunities 
for openness that flow from their thoughtful application, in the form of availability 
of and access to formal and informal learning (Ferguson, Jones, & Scanlon, 2019; 
Littlejohn & Margaryan, 2014). Without research to inform practice the changes in 
learning may exclude the very people who most stand to benefit from them. 
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For example, those likely to complete free, online courses tend to be qualified to 
degree level already (Kizilcec, Saltarelli, Reich, & Cohen, 2017; Rizvi, Rienties, 
Kizilcec, & Rogaten, 2022). Ironically, the revolution in open world learning is in 
danger of increasing the digital divide by privileging those with the appropriate 
digital and learning skills to best take advantage of it (Iwaniec-Thompson, 2022; 
Nguyen, Rienties, & Richardson, 2020). It is this issue that this edited book on 
open world learning will address. 

The complex (Adams, Fitzgerald, & Priestnall, 2013; Richardson, Mittelmeier, & 
Rienties, 2020), contradictory (Bayne & Land, 2013; Nguyen, Rienties, & 
Richardson, 2020; Weller, 2020), and multidisciplinary (Ferguson et al., 2019; 
Hampel, 2014; Mittelmeier, Rienties, Gunter, & Raghuram, 2020; Scanlon, 2014) 
nature of open world learning is fundamentally changing society and the founda- 
tions of education. This drives an urgent need to review the “enablers” and “dis- 
ablers” of open world learning for inclusive approaches to learning across 
educational sectors, disciplines, and countries (Barber, 2021). 

The world is becoming a more connected place with the emergence of immediate 
access systems, such as smartphones and tablets (Kukulska-Hulme et al, 2020; 
Srisontisuk, 2022), the omnipresence of social media like Facebook (Vogiatzis, 
Charitonos, Giaxoglou, & Lewis, 2022), Twitter (Bruguera, Guitert, & Romeu, 2019; 
Rehm, Cornelissen, Notten, Daly, & Supovitz, 2020), WhatsApp (Madge 
et al., 2019; Vogiatzis et al., 2022), and new methods of working (Bond, Zawacki- 
Richter, & Nichols, 2019; Bruguera et al., 2019; Kukulska-Hulme et al., 2020; Lucena, 
Diaz, Reche, & Rodriguez, 2019). This is accompanied by the advent of English as a 
lingua franca that is helping people to share information and communicate across the 
globe (Conde Gafaro, 2022; Hampel, 2014; Rets, Stickler, Coughlan, & Astruc, 2022; 
Rienties, Lewis, O'Dowd, Rets, & Rogaten, 2020;Vogiatzis et al., 2022). 

Open world learning seeks to understand access to education, structures, and the 
presence of dialogue and support systems (Iniesto & Hillaire, 2022; Mittelmeier 
et al., 2020; Weller, 2020). The challenge of open world learning is to foster inclu- 
sion and widen access to information, knowledge, and learning, rather than to 
allow people, organisations, and governments to increase divisions and build closed 
groups with privileged access to information and education. One key aspect of 
open world learning is openness: how approaches to learning can be designed to 
enable sharing and co-creation of knowledge (Iniesto, McAndrew, Minocha, & 
Coughlan, 2022; Littlejohn et al., 2019; McAndrew & Scanlon, 2013; Mohamud, 
Buckler, Pitt, & Twining, 2022; Rizvi et al., 2022). However, these innovations are 
also changing how societies understand ownership of knowledge, information, 
expertise, and the process of learning. 

Nonetheless, openness also has its disablers (e.g., restrictive quality assurance 
regimes, required integration across units, accreditation and costs), and the 
excluding power of “closed” systems (such as classrooms, universities, or corpo- 
rate training programmes) may seem more persuasive in some situations (Ferguson 
et al., 2019; Korir, Mittelmeier, & Rienties, 2020). Many open technologies are 
subject to — and still have in many cases — (over)inflated expectations as change 
agents (e.g., artificial intelligence-enabled learning, virtual world learning) whilst 
only providing different practices within the same educational parameters (e.g., 
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in the context of a transmission-based approach to learning) (Bond et al., 2019; 
Neelen & Kirschner, 2020; Rienties, Kohler Simonsen, & Herodotou, 2020). Some 
of these “technical disruptions” actually contribute to maintaining restrictive edu- 
cational norms and practices (Bayne & Land, 2013; Nguyen, Rienties, & Whitelock, 
2022). This is, for example, the case with many MOOCs, which, despite their 
aspirations to fundamentally open up education, are not only based on a 
transmission-based paradigm but also used mainly by well-educated learners in 
Western countries (Kizilcec et al., 2017; Rizvi et al., 2022). 

Similarly, there are ethical and privacy implications, for example, in learning ana- 
lytics through increasing levels of monitoring, surveillance, and profiling (boyd & 
Crawford, 2012; Korir et al., 2020; Korir, Slade, Holmes, & Rienties, 2022), or public 
concerns whether open education will shift the balance of power of governments, 
schools, and educational institutions to a global, uncontrolled space. Finally, several 
groups of users (e.g., teachers, older people, people with particular accessibility needs) 
seem to struggle to embrace and integrate open technology, leading to resistance and 
anxieties towards new technologies (Bruguera et al., 2019; Iniesto et al., 2022; 
Iwaniec-Thompson, 2022; Nguyen, Rienties, & Whitelock, 2020). 


1.2 Moving from practice to theory (and back to 
practice) 


At the moment, open world practice is leading theory (Gasevic, Dawson, Rogers, 
& Gasevic, 2016; Herodotou et al., 2019; Neelen & Kirschner, 2020; Weller, 2020), 
and research is needed to close this gap and allow experiences from particular con- 
texts to inform generalised approaches with strong conceptual underpinnings. This 
ground-breaking and world leading book is a result of an award-winning (Open 
Education Consortium 2016) Leverhulme doctoral training programme that was 
structured to generate evidence, encourage theory construction, and lead to well- 
described new knowledge that informs practice across disciplines from 18 PhD 
students at The Open University, UK (OU).The OU has been on the forefront of 
continuous innovation in open and distance education for over 50 years (Barber, 
2021; Ferguson et al., 2019; Lucena et al., 2019; Weller, 2020). 

The main question of this book is as follows: How can open world learning 
supported by technology help and/or hinder tackling the global chal- 
lenges that open and high-quality education faces? First, this book provides 
an integrated and cohesive perspective of the affordances and limitations of open 
world learning. We strive to build a bridge that connects a range of research com- 
munities (e.g., artificial intelligence, computing, educational psychology, HCI, 
language education, learning analytics, learning sciences, linguistics, Open 
Educational Resource) that draw theoretically, conceptually, and analytically from 
each other, but have not always engaged in discussions to learn from each other’s 
perspectives. Second, this book features a wide range of open world learning top- 
ics, ranging from theoretical and methodological discussions to empirical demon- 
strations of how open world learning may be actually implemented, evaluated, 
and used to inform theory and practice. Furthermore, this book will provide in- 
depth analyses of the potential benefits and limitations of open world learning by 
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bringing together insights from 387,134 learners, practitioners, and educators 
working and learning in 136 unique learning contexts (e.g., online courses, 
MOOCG;, Internet kiosks). 


1.3 Rationale and structure of this book 


This book will provide state-of-the-art contemporary research insights of key sys- 
tems, applications, and processes in open world learning. There is a need to raise 
awareness of academics, researchers, professionals, and policymakers regarding the 
affordances and limitations of technologies and approaches related to open world 
learning, and how this influences daily practice around us. This book provides new 
and substantial findings from 41 leading and promising early-career researchers and 
academic supervisors from 13 institutions in open world learning (for a full list, see 
list of contributors). Furthermore, the chapters focus on a range of countries, 
including Greece, Italy, the Netherlands, Uganda, and the UK, while five chapters 
obtained data and perspectives from across the globe. 

Although we have developed this book as a collaborative project with a particu- 
lar and hopefully logical structure, each chapter in itself can be read as an individual 
piece of academic work. Through clear referencing throughout the book, inter- 
ested readers can delve into specific sections or chapters or read the book in a 
non-linear manner. Nonetheless, here is a brief overview of the book. 

After a general introduction and overview of the affordances and limitations of 
open world learning by using both a macro-meso-micro and 4P lens (i.e., people, 
places, practices, properties) in Chapter 2 (Rienties, 2022), in Part 1 Learners 
and the power of language in an open world, we provide six chapters on how 
learners across the globe can use the power of English language to communicate 
together in an open world. The five chapters in Part 2 Innovative technologies 
in an open world provide insight into innovative technologies that have been 
implemented and evaluated in an open world, including internet kiosks, MOOCs, 
online games, and sentiment mining. Finally, in Part 3 Educators and inclusive 
practice in an open world, we specifically look at the role of educators in sup- 
porting inclusive practices in an open world. In the remainder of this chapter, an 
overview of each of the 19 chapters provides you, the reader, the opportunity to 
identify which Part(s) and/or Chapter(s) may be most relevant for you. 


1.3.1 Part | Learners and the power of language in an open 
world 


A common language is essential for learning in an open world and for learners to 
be able to communicate with each other. Part 1 comprises six chapters on how 
learners across the globe can use the power of (English) language to communicate 
together in an open world. Three chapters focus on adult learners (Conde Gafaro, 
2022; Rets et al., 2022;Vogiatzis et al., 2022), two focus on MOOC learners (Chua, 
2022; Rizvi et al., 2022), and one on children (Anastasiou, 2022). Furthermore, a 
mix of methodological approaches is used to unpack the complexities of language 
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learning in an open world, including qualitative (Anastasiou, 2022) and multiple/ 
mixed-methods (Chua, 2022; Conde Gafaro, 2022; Rets et al., 2022; Rizvi et al., 
2022; Vogiatzis et al., 2022). Finally, a mix of digital and online tools in an open 
world learning context are used in these chapters, including Futurelearn (Chua, 
2022; Conde Gafaro, 2022; Rizvi et al., 2022), Open Educational Resources (Rets 
et al., 2022), WhatsApp (Vogiatzis et al., 2022), and You Tube (Anastasiou, 2022). 

In Chapter 2, Popi Anastasiou (2022) provides a rich and detailed study of 31 Greek 
primary-school children who in an experimental design worked together in small 
groups on developing digital stories in science in either a story sequencing activity or 
story verbalisation activity. The findings indicate that story sequencing allowed chil- 
dren to position themselves as story makers, who employed decision-making strate- 
gies to determine the content of the story according to their preferred order of 
presentation. In contrast, in the story verbalisation groups, children were acting as 
story re-tellers, who mainly verbalised pre-defined content. Chapter 3 highlights how 
children from an early age onwards can creatively use a range of open world learning 
tools to tell their own stories and make meaning in their own language. 

In Chapter 4, Dimitrios Vogiatzis et al. (2022) explore how WhatsApp can be 
effectively used for language learning by adults in a German context. Using a case 
study design with online observations and semi-structured interviews, the authors 
explore group interactions between eight learners over 22 weeks. The findings 
indicate adult learners! reactive participation, and mostly limited interaction, 
emphasising that the mere use of WhatsApp cannot necessarily guarantee language 
interaction among participants. This is an important finding as just having a com- 
mon open world learning tool does not necessarily imply that all learners will be 
able to successfully interact and learn with these kinds of tools, and points to the 
crucial area of learning design explored later in this book. 

In Chapter 5, Barbara Conde Gafaro (2022) examines the self-regulated learn- 
ing behaviours of 19 language learners who engage with MOOCs as part of a 
blend with their language learning courses. Using a mix of thematic analysis and 
measurements of self-regulated learning, the findings indicate that learners not 
only set short-term goals but also set multiple reflexive learning goals as part of 
their MOOC experience. Supporting learners in the process of goal setting may 
contribute to increasing their engagement with blended language courses. 

In Chapter 6, Shi Min Chua (2022) uses an innovative corpus linguistic approach 
to understand how learners communicate in large open spaces like MOOCs. In a 
big data exploration of online discussions containing 11 million word contribu- 
tions from 228,665 learners in 12 Futurelearn MOOCS, the findings indicate how 
learners use particular communicative strategies can significantly influence whether 
(or not) other learners are going to reply to a message, and how conversations can 
be made meaningful when they reply to each other, without causing hard feeling 
but making it an open discussion. Chapter 6 highlights again the importance of 
language use and suggests several useful writing tips for learners who would like to 
open a conversation with others in online discussion. 

In Chapter 7, Irina Rets et al. (2022) explore how Open Educational Resources 
(OERs) could be made more accessible for non-native English speakers using a 
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mixed method of eye-tracking with simulated recall interviews with nine partici- 
pants. The findings from in-depth analyses with nine participants indicate that 
decreasing the complexity level of OERs by text simplification led to a reduction 
in cognitive load and an increase in higher-level processing. Given the global reach 
of online learning tools and educational resources, this chapter highlights that 
teachers need to carefully balance their language usage to make it accessible for 
both native and non-native language speakers. 

Finally, in Chapter 8 Saman Rizvi et al. (2022) provide two studies to explore 
whether or not MOOCCS are culturally inclusive. The first study uses a quantitative 
approach to investigate the extent to which engagement by 49,582 learners in ten 
Futurelearn MOOCs varied between geo-cultural contexts, while the second fol- 
low-up study examines the link between learners’ geo-cultural background and 
their perspectives about how different elements of learning design contribute 
towards their learning. The findings suggest that learners from different geocultural 
backgrounds substantially differed in their engagements in MOOC activities (e.g., 
articles discussions, videos), while the qualitative findings highlighted large per- 
ceived cultural differences in the value of certain MOOC activities, which in part 
were influenced by participants English language abilities. This further highlights 
the need for learners and teachers to recognise that while open world learning 
approaches are theoretically open to all, in practice they are not necessarily inclu- 
sive for anyone and everyone. 


1.3.2 Part 2 Innovative technologies in an open world 


Part 2 contains five chapters focusing on innovative technologies that have been 
implemented and evaluated in an open world context. Two chapters focus on stu- 
dents (Hillaire, Rienties, Fenton-O’Creevy, Zdrahal, & Tempelaar, 2022; Korir et 
al., 2022), two use a multi-stakeholder perspective (Iniesto et al., 2022; Mohamud 
et al., 2022), and one focuses on gamers (Hall, Herodotou, & Iacovides, 2022), 
Furthermore, a mix of methodological approaches is used to unpack the complexi- 
ties of implementing innovative technologies in an open world, including qualita- 
tive (Mohamud et al, 2022), quantitative (Hall et al, 2022) and multiple/ 
mixed-methods (Hillaire et al., 2022; Iniesto et al., 2022; Korir et al., 2022). Finally, 
like in Part 1 a mix of open world learning tools are used in these chapters, includ- 
ing digital games (Hall et al., 2022), Futurelearn (Iniesto et al., 2022), solar-powered 
internet kiosks (Mohamud et al., 2022), Prolific (Korir et al., 2022), and a student 
sourced sentiment analysis classifier (Hillaire et al., 2022). 

In Chapter 9, Francisco Iniesto et al. (2022) explore the accessibility in MOOCs via 
interviews with 40 MOOC providers designers and 34,026 learners using a mixed 
method of survey data and an accessibility audit. The findings indicate that while sub- 
stantial progress has been made over the years to improve accessibility in MOOCs, 
substantial barriers remain, and at times rather than making designs inclusive some 
providers have restricted access. Chapter 9 concludes with a call for inclusive design 
by adding in a consideration of specific learner groups so that they are included, 
potentially through alternative design solutions rather than one design solution. 
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In Chapter 10, Khadija Mohamud et al. (2022) illustrate the impact of solar- 
powered internet kiosks on 50 users in two urban low-income communities in 
Uganda. The findings highlight how users utilised these centres to empower 
themselves beyond the use of technology. Specifically, this is related to supporting 
young people and teachers with instructional resources to bridge gaps in education 
and providing low-income communities with a social space where they nurtured 
meaningful relationships cultivating their sense of belonging. Furthermore, 
Chapter 10 illustrates how the technology allowed others to learn and also raised 
some safety concerns. 

In Chapter 11, Maina Korir et al. (2022) explore how the crowdsourcing plat- 
form Prolific could be used to elicit students’ data use preferences for learning 
analytics. In an experimental design, 447 students from different higher and further 
education institutions in the UK were presented with privacy risks and/or benefits 
interventions to examine whether and how these would influence their data use 
preferences. The findings indicate that these interventions did not substantially 
alter participants’ privacy notions and support of institutional use of student data. 
Participants’ responses indicated that they made trade-offs to arrive at what was an 
acceptable use of student data for them. 

In Chapter 12, Johanna Hall et al. (2022) explore how creativity in digital games 
plays a role in openness to experience, emotional wellbeing, and meaningful learn- 
ing. By developing a Creativity in Gaming Scale (CGS) instrument based upon the 
input from 251 gamers, the authors provide a powerful tool for teachers and learn- 
ers to explore whether their digital games encourage creativity, including problem- 
solving and appropriation. 

Finally, in Chapter 13, Garron Hillaire et al. (2022) explore how student voices 
and inputs can be used to develop a student sourced sentiment analysis classifier. 
While most off-the-shelf sentiment analysis tools are developed outside education 
and educational context, by analysing online contributions by 1,251 students at 
one university in the Netherlands to an online collaborative experiment and after- 
wards asking them to code their own and others emotions, an inclusive classifier is 
developed. The findings indicate that this classifier is able to accurately identify 
emotions in online chat. 


1.3.3 Part 3 Educators and inclusive practice in an open world 


Finally, in Part 3, we look specifically at the role of educators, teachers, and profes- 
sionals in supporting inclusive practices in an open world. Two chapters specifi- 
cally focus on how teachers make learning design decisions (Iniesto & Hillaire, 
2022; Nguyen et al, 2022), one focuses on teachers in early years contexts 
(Srisontisuk, 2022), one on finance professionals (Chaudhari, Littlejohn, & Cross, 
2022), and one on older academics (Iwaniec- Thompson, 2022). Furthermore, 
again a mix of methodological approaches is used to unpack the complexities of 
how educators and professionals make sense of an open world, including qualita- 
tive (Chaudhari et al., 2022; Iniesto & Hillaire, 2022; Iwaniec- Thompson, 2022; 
Srisontisuk, 2022) and multiple/mixed-methods (INguyen et al., 2022). Finally, 
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like in Part 1 and Part 2, a range of open world learning tools is used in these 
chapters, including FutureLearn (Iniesto & Hillaire, 2022), learning analytics 
(Nguyen et al., 2022), and Ipads (Srisontisuk, 2022). 

In Chapter 14, Quan Nguyen et al. (2022) explore how the use of data on how 
teachers design their courses can inform our understanding of students! engage- 
ment with online learning activities. By mapping both 70,000 students and 39 
teachers' data, this data-driven approach provides teachers with the opportunity to 
reflect on their course design through visualisations of weekly learning activities 
which highlights the workload and the variety of teaching approaches. 

In Chapter 15, Francisco Iniesto and Garron Hillaire (2022) apply a Universal 
Design for Learning (UDL) approach to map how effective a range of MOOCs 
from different providers are in terms of accessibility. When producing educational 
resources, all stakeholders should be considered in their design and evaluation pro- 
cesses including educators and learners. The iterative and refinement processes 
shown in Chapter 15 indicate that UDL, as a proactive design framework, can be 
used for evaluation, and facilitate and help with internalisation of its principles. 

In Chapter 16, Pinsuda Srisontisuk (2022) explores the rich and diverse perspec- 
tives of nine UK teachers on smartphone and tablets technologies in two classrooms. 
The findings indicate most children can use tablets with minimal instruction, 
although there remains a fear amongst teachers around the technology leading to an 
increase in learners being passive and not engaging in other social and physical activ- 
ity. Nonetheless, the teachers also reported that one of the most beneficial features 
was the connectivity to the world wide web; instant access to knowledge and images 
was a feature they felt has enhanced the learning experience for the children. 

In Chapter 17,Vasudha Chaudhari et al. (2022) explore how nine professionals 
in the finance sector make sense of uncertainty. The findings suggest that although 
uncertainty is experienced by all professionals irrespective of their level of exper- 
tise, there are distinct differences in the way uncertainty is perceived by experts and 
practitioners. Chapter 17 presents the similarities and differences in perception of 
uncertainty and discusses the implications of these findings for practice. 

Finally, in Chapter 18, Gosia Iwaniec- Thompson (2022) explores how eleven 
older academics are conceptualising their identity development and use of tech- 
nology in particular. The manifestation of subjectivities occurs when academics 
exert influence and take stances affecting their practice and impact on the con- 
strual of their professional identity. For some, open world tools like ResearchGate 
were helpful for developing their identities, while other tools led to uncertainty 
among the academics of how to integrate these into their teaching practice. 


1.4 Discussion 


In this book, we will bring together some diverse perspectives and narratives 
around open world learning from 387,134 learners and teachers working in 136 
learning contexts. As highlighted in the final Chapter 19 (Rienties, Hampel, 
Scanlon, & Whitelock, 2022) and throughout this book, while open world learning 
approaches, methods, and tools provide many affordances for learners, teachers, and 
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professionals, not everyone is able to necessarily benefit from these just because 
these are “open”. In this book and in Chapter 19, we suggest a range of practical 
steps that you as reader can do to ensure that the power of open world learning is 
as inclusive as possible. 
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Chapter 2 


Powers and limitations of open 
world learning 


Experiences from the field of education 


Bart Rienties 


2.1 Introduction 


In September 2014, an interdisciplinary group of 18 researchers from four Faculties 
at the Open University UK (OU) submitted a (successful) £1 million large grant 
proposal to the Leverhulme Foundation entitled Open World Learning. In its pro- 
posal,' the group indicated that 


[w]e seek support for a programme of PhD studies to address inclusive 
approaches to learning across disciplines, integrated by a focus on Open World 
Learning, for which the OU has a worldwide reputation. Our proposal aims 
to increase understandings of the complexities of Open World Learning com- 
pared to systems and approaches that close rather than open opportunities 
because of social, geographical, or technical barriers. To do this requires a 
strong cross-disciplinary approach centred on educational thinking but involv- 
ing a range of subjects across the University. This cross-disciplinarity means 
that obtaining funding from domain-specific research councils is not straight- 
forward and there is no targeted programme in this area. The Leverhulme 
Foundation's Doctoral Scholarship funding, on the other hand, is specifically 
focussed on cross-disciplinary and complex research topics. 


As already described in Chapter 1 (Rienties, Hampel, Scanlon, & Whitelock, 
20222) and argued throughout this book, learning in the 21st century is under- 
going both subtle and radical transformation as a result of the impact of digital, 
networked technologies (Bond, Zawacki-Richter, & Nichols, 2019; Ferguson, 
Jones, & Scanlon, 2019; Hampel, 2019). Open learning gives unprecedented 
access to knowledge, information, and education and provides support to learn- 
ers across the globe (Kizilcec, Saltarelli, Reich, & Cohen, 2017; Kukulska-Hulme 
et al., 2021; Weller, 2020). However, it is not the technologies themselves that 
represent the biggest change, but the opportunities for openness that flow from 
their thoughtful application, in the form of availability of and access to formal 
and informal learning (Iniesto, McAndrew, Minocha, & Coughlan, 2022; Rizvi, 
Rienties, Kizilcec, & Rogaten, 2022; Weller, 2020). Without research to gain 
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deeper understanding, the changes in learning may exclude the very people who 
most stand to benefit from them. 

Within the Leverhulme Open World Learning programme, 18 scholars from 16 
different countries have each contributed to understanding how the enablers and 
disablers of open world learning might have shifted over time, and how each shift 
might have had expected and unexpected consequences in open world learning. In 
this chapter, I critically review some of the key events that have shaped our open 
world learning in the period 2014—2022, and how this might potentially develop 
over time in a UK, European, and perhaps global context. 

This will by no means be an accurate reflection and review of all facts, perspec- 
tives, and contexts. I am acutely aware that experiences and reflections are strongly 
embedded in one’s own context (i.e., working in a top UK distance learning 
organisation, being a white male academic from Europe, having access to nearly 
unlimited open world learning opportunities). Nonetheless, I hope that by sharing 
some of my experiences and reflections of working with leading Leverhulme 
scholars as well as working with brilliant and state-of-the-art researchers across 
four Faculties at the OU will help to make sense of how leading institutes like the 
OU are trailblasing the way to make sense of open world learning. 


2.2 Making sense of open world learning on a 
macro-meso-micro level 


As conceptually visualised in Figure 2.1, there are both enablers and disablers for 
open world learning. The fluidity in the 3D shape or space of the open world 
learning framework highlights that the size and space of open world learning expe- 
rienced by individual learners and organisations in a particular context, region, or 
nation might be substantially different from other learners and organisations. 
Furthermore, the fluidity of the 3D shape and space will inevitably change over 
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Figure 2.1 Open world learning framework (2014). 
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time, making the reach, depth, and breadth of open world learning experiences 
more or less comprehensive. For example, those learners who have no digital 
devices, no internet access, and/or who live in countries with very strict access 
policies might at best only experience open world learning in a limited, match-box 
manner. In Chapter 10, Mohamud, Buckler, Pitt, and Twining (2022) provide a 
fascinating account of how providing internet kiosks in Uganda opens up a win- 
dow of opportunity for open world learning. Other learners who have ubiquitous 
access, competences, skills, and resources to explore open world learning opportu- 
nities and who live in a very open country might have nearly infinite opportunities 
to enable experience and potentially benefit from open world learning. 

At the same time, with the growth in access to online education and the 
increased competition amongst educational providers and arrival of new commer- 
cial entrants into the market, there is also an expansion of the "closed" learning 
industry (i.e., paid-for by learners and/or organisations) by a range of diversifica- 
tion and growth strategies (e.g., apprenticeships, dual degrees, micro-credentials, 
short-courses). Obviously, there might be trickle-down effects from closed learn- 
ing to open learning (e.g., free open short courses as spin-offs from closed courses), 
as well as opportunities for pathways from open learning into formal degree rec- 
ognition (e.g., using accreditation of MOOC competition to a formal degree). In 
other words, the “forces” of the enablers and disablers of open world learning hold 
each other together in a (temporary) equilibrium, whereby a shift in any of the 
building blocks might change the equilibrium. 

In this conceptual model, I distinguish three levels or spheres of influence on a 
macro, meso, and micro level. Note that the colouring of these boxes in Figure 2.1 
was based upon our joined understanding of the literature and practice within an 
OU context on each of these levels in 2014. More pronounced darker colours 
indicate a stronger evidence of experience and research, while lighter boxes repre- 
sent relatively uncharted research areas. 


2.2.1 Macro 


On a macro (i.e., regional, national, international, global) level, many things have 
changed since 2014 and will continue to change. In Chapter 2, I highlight three 
examples of macro changes in 2014—2022 that inevitably have changed the open 
word learning spaces for large groups of learners, namely technological change and 
development, rise of populism, and Brexit. First of all, technological change and devel- 
opment is continuous. For example, in 2014 the first voice-activated virtual assistant 
in households was introduced (Alexa) by Amazon, whereby in 2020, 22% of UK 
households have such a smart speaker in their household (Ofcom, 2020). In 2019, 
5G was launched in the UK giving users over a hundred times faster internet on 
their phones than in 2012, and uptake in terms of smartphones increased from 61% 
in 2014 to 82% in 2020 (Ofcom, 2020). 

While each of the many technologies introduced since 2014 provide their own 
affordances and limitations, in the last seven years how learners can access online 
resources has radically accelerated. Indeed, a recent Ofcom (2020) report indicated 
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that in the UK during lockdown, on average people spent a record 6 hours and 
25 minutes per day watching audiovisual content, often on Smart TVs, and spent 
on average 4 hours and 2 minutes online, mostly via smartphones (Ofcom, 2020). 
On the one hand, with most Western jobs being desk-oriented jobs and requiring 
staff to sit behind a computer, many people have unprecedented access to screens 
and the internet, thus potentially allowing for opportunities for open world learn- 
ing. On the other hand, the unprecedented access to screens may negatively 
impact mental health and wellbeing (Lucassen et al., 2018), and learners’ willing- 
ness to engage with online learning. Indeed, a lot of organisations (Wellcome 
Trust, 2020) and the public sector (World Health Organization, 2019) are con- 
cerned about the unprecedented access to information and technology, and the 
increase in screen time. Anecdotally, several CEOs of large tech companies includ- 
ing Microsoft and Snapchat restrict the screen time of their children or even 
forbid them to use technology. 

A second major macro-development is the rise of populism in many Western 
countries. Narratives around post-truth and #fakenews are continuing to have a 
large impact on the perceptions of social media (Ernst, Engesser, Biichel, Blassnig, 
& Esser, 2017; Isaak & Hanna, 2018), and in particular the perceived value of 
(higher) education (Quinton, 2019) and recognition of experts (Bruggeman et al., 
2020; Kubin, Puryear, Schein, & Gray, 2021). A range of studies have started to 
explore how people make sense of these complexities and how learners make sense 
of open knowledge and information. For example, a large-scale study amongst 389 
US domestic students by Quinton (2019) showed that students who were more 
conservative, supported (former) President Trump, and had more negative stereo- 
types towards international students were significantly less likely to interact with 
other (international) students. In cross-cultural literature, it is a well-established fact 
that social interaction between people from different cultures helps to overcome 
stereotypes and allows people to develop mutual relations and understanding (Jing, 
Ghosh, Sun, & Liu, 2020). 

Furthermore, in a lab-based study with 308 first-year business students, Knight 
et al. (2017) found that while most participants indicated to have strong internet 
searching skills, their actual searching strategies and behaviour indicated that par- 
ticipants mainly used lay health advice website sources rather than academic 
sources to solve a complex health case. Similarly, using in total 15 studies, Kubin 
et al. (2021) found that personal experience about a (political) issue, in particular 
negative experience, was weighted more important than having expertise, knowl- 
edge, facts, or data. In other words, while many learners have unprecedented 
access to knowledge and information, substantial development and training will 
be needed to ensure that learners can develop strong epistemological skills to 
make sense of open world learning in this post-truth era. 

A third macro trend for the UK in particular is that recent political develop- 
ments have partially undone 40 years of intensive and open collaboration between 
European and UK companies, public organisations, and research institutions, and 
has introduced a range of new barriers in terms of how learners can access data, 
study, travel, work, etc. In a way, this unexpected macro development went against 
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the grain of increased globalisation and open world learning. In part the unex- 
pected referendum result was blamed on social media, and the impact of Cambridge 
Analytica in particular (Isaak & Hanna, 2018). In part concerns about international 
migration and pressure on British identity and values fuelled by (social) media 
(Gavin, 2018), post-truth, and a divide amongst voters mainly along (higher) edu- 
cation lines (Zhang, 2018) heated opinions. On the one hand, one could argue that 
open world learning approaches allowed more people to gain access to knowledge, 
skills, and facts. On the other hand, social media was substantially influenced by 
political and external parties, and individual (negative) experiences, which may 
have influenced the referendum result. Perhaps it was not a surprise that the day 
after asking whether the UK should leave the EU the second most searched ques- 
tion in Google UK was “What is the EU?" (Fung, 2016). 


2.2.2 Meso 


On a meso (i.e., institutional, cross-institutional, cross-discipline) level, education 
has experienced some radical changes in the last seven years, not only in the UK 
but worldwide. For example, there is an ever-increasing shift towards consumerism 
of education (Bragg, 2014; Langan & Harris, 2019), as well as stronger competition 
and managerialism within and across institutions (Erickson, Hanna, & Walker, 
2020). Furthermore, there are increased pressures on teachers to include technol- 
ogy in teaching (Bond et al., 2019; Herodotou, Rienties, Boroowa, & Zdrahal, 
2019; Uerz, Volman, & Kral, 2018; van Leeuwen, 2019), leading to potential burn- 
out and mental stress amongst some teachers (Daniel, 2018), researchers (Wellcome 
Trust, 2020), and students (Houghton & Anderson, 2017). All these trends were 
accelerated and exacerbated by the COVID-19 pandemic, where teachers sud- 
denly had to become experts in blended/online teaching overnight (Crawford 
et al., 2020; Naffi, 2020; Reinholz, Stone-Johnstone, White, Sianez, & Shah, 2020). 

Another meso change is the increased focus on Equality, Diversity and 
Inclusion (EDI) in (higher) education. In part this increased focus on EDI is 
fuelled by the #metoo and #blacklivesmatter movements that have a macro and 
meso impact, as well as a range of studies highlighting substantial inequality in 
opportunity in education (e.g., Bhopal & Henderson, 2021; Lucassen et al., 2018; 
Nguyen, Rienties, & Richardson, 20202; Richardson, Mittelmeier, & Rienties, 
2020). For example, in Chapter 15, Iniesto and Hillaire (2022) indicate that 
despite efforts by MOOC providers to make their materials accessible, in practice 
this often falls short for learners with accessibility needs. Similarly, in Chapter 8, 
Rizvi et al. (2022) show that despite best intentions, MOOC learning activities 
are not necessarily fit for purpose for some geo-cultural groups of learners, as the 
(Western) learning designs of MOOCs do not necessarily fit with their preferred 
learning approach. 

Indeed, a range of institutional and cross-institutional initiatives such as Athena 
Swan and Race Equality Charter have started to address some of these inequalities 
of opportunities, including explicit requirements for institutions to report on how 
they are addressing these inequalities. However, there are substantial tensions in 
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terms of prioritising which part of EDI to focus on first, as argued by Bhopal 
and Henderson (2021, p. 167) "[w]hile institutions can claim to be working on 
structural inequality by focusing time, resources and attention on gender equal- 
ity, there is little or no imperative to shift the focus to uncomfortable conversa- 
tions about race and racism in the academy". Both the macro and meso changes 
in the last seven years have had a substantial impact on the micro level of open 
world learning. 


2.2.3 Micro 


Finally, on a micro (within institution, module, student) level, several large 
changes have occurred that have influenced how educators design courses and 
how learners learn. For example, with the omnipresence of technology and 
mobile devices in the classroom, the way students learn and interact with others 
is rapidly changing education (Kukulska-Hulme et al., 2021; Srisontisuk, 2022). 
For example, in the OU, while many courses in 2014 had several offline and 
print-based learning resources, in 2021, nearly all resources are primarily avail- 
able online (Nguyen, Rienties, & Whitelock, 2020b). Similarly, with the increased 
availability of free online resources, MOOCs, and other learning opportunities, 
students and teachers have unprecedented access to knowledge and information 
(Conde Gafaro, 2022). 

A large change on a micro-level is the access to data and learning analytics data 
in particular. While in 2014 mostly institutions were storing some data for retro- 
spective reporting for accreditation processes and government bodies, an increas- 
ing number of teachers and students have gained (near) real-time access to 
learning and learner data. For example, Herodotou et al. (2020) analysed how the 
predictive learning analytics tool OU Analyse was accessed by 1159 unique 
teachers and reached 23,180 students in 231 undergraduate courses in the last 
four years, and found substantial different adoption patterns between teachers. 
These in part were explained by how Faculties engaged with predictive learning 
analytics, how they encouraged teachers as “champions”, as well as teachers’ digi- 
tal literacy and their conceptions about teaching online. In Chapter 13, Hillaire, 
Rienties, Fenton-O’Creevy, Zdrahal, and Tempelaar (2022) show that based 
upon online chat discourse between 1251 business undergraduate students, a 
student-sourced sentiment analysis tool can be developed that can accurately 
predict emotions in written text. 


2.3 The 4P approach of Open World Learning 


As indicated in Figure 2.1, beyond the macro-meso-micro levels, the 4P approach 
distinguishes between four important inter-related themes that may influence the 
enablers and disablers of open world learning: people, places, practices, and proper- 
ties. Given the nature of open world learning, some of these enablers or disablers 
of learning might influence the four themes differently. 
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2.3.1 People 


In terms of the theme people accessibility needs (Iniesto et al., 2022), socio- 
economic factors (Rizvi et al., 2022), and personal learner characteristics such as 
age (Iwaniec- Thompson, 2022), ethnicity (Nguyen, Rienties et al., 2020b), gender 
(Richardson et al., 2020), and learning dispositions (Tempelaar, Rienties, & Nguyen, 
2021) might influence whether people can benefit from open-world learning. For 
example, in Chapter 7, Rets et al. (2022) show that Open Educational Resources 
(OERs) could be made more accessible for non-native English speakers by decreas- 
ing their complexity level by text simplification. In Chapter 14 Nguyen, Rienties 
and Whitelock (2022) shows that teachers make complex decisions when designing 
closed and open world learning designs, which in part are influenced by their own 
conceptions of teaching, as well as the culture of the discipline in which they work. 


2.3.2 Places 


Secondly, places where people work or live might substantially impact on how they 
get access to open world learning opportunities. For example, Facebook restricted 
access to news sites in Australia in February 2021, thereby limiting people's access 
to knowledge and information. In some countries like Myanmar or Belarus, 
Facebook and other social media sites have been taken offline during social unrest, 
while in other countries certain topics are censored or even completely barred. 
Beyond actively restricting content and access to open world learning, even when 
learning activities are openly available this does not necessarily imply that users will 
universally make sense of them in the same way. For example, in Chapter 8, Rizvi 
et al. (2022) show in two studies that MOOCs are not necessarily culturally inclu- 
sive, based upon engagement patterns by 49,582 learners in ten Futurelearn 
MOOGC-s. In Chapter 4,Vogiatzis et al. (2022) explore how WhatsApp was used by 
German-language learners on the move, while in Chapter 10, Mohamud et al. 
(2022) show how internet kiosks in one particular context of Uganda raised some 
culturally specific opportunities and concerns. 


2.3.3 Practices 


Thirdly, the practices people and institutions are surrounded by might influence 
how they engage with open world learning. Although sometimes we seem to live 
in a global village, the way that education is shaped in countries is mostly deter- 
mined by policy makers and stakeholders on a national level, whereby consistently 
substantial differences in practices are noticed on a regional, national, and geo- 
cultural level (Kizilcec et al., 2017; OECD, 2018; Rizvi et al., 2022). Also, girls are 
particularly likely to be left behind (Girls! Education Challenge, 2021). An emerg- 
ing body of research has shown that for example while MOOCs are accessible 
across the globe, some nations and geo-cultural regions are more likely to benefit 
from these open world learning opportunities than others. Indeed, accessibility 
policies of MOOCs are framed and shaped by local and national practices and 
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policies as evidenced in Chapter 9 (Iniesto et al., 2022). In the context of financial 
sectors, in Chapter 17, Chaudhari, Littlejohn, and Cross (2022) show that how 
finance professionals cope with uncertainty and make use of technologies is nested 
within their practice. 


2.3.4 Properties 


Finally, properties of open world learning technologies and data will substantially 
influence the affordances and limitations of how people make sense of open world 
learning. As illustrated in Figure 2.1, there was a relative paucity of research in 2014 
on how properties of data and practices within open world learning might enable 
or thwart learning. For example, the OU was the first institution to introduce an 
ethics policy on the use of data with learning analytics (Open University UK, 
2014). In Chapter 11, Korir et al. (2022) show that most UK learners are now rea- 
sonably comfortable to share their data with higher education institutions, while 
initially some authors reported concerns about privacy and sharing of data. In 
Chapter 12, Nguyen et al. (2022) show that OU teachers make substantial use of 
student engagement data to determine how they design online courses. In particu- 
lar how they balance assessment, interactive, and communicative learning activities 
seems to significantly impact student engagement, retention and satisfaction. At the 
same time, with the triangulation of more and more data and as Artificial 
Intelligence is becoming more intertwined in education (Rienties, Kohler 
Simonsen, & Herodotou, 2020), there are substantial concerns about how algo- 
rithms are potentially making decisions that could influence behaviour and perfor- 
mance of learners (Baker & Hawn, 2021; Prinsloo & Slade, 2017). 


2.4 Discussion and moving forward 


In this book, each of the chapters will explore how the enablers and disablers of 
open world learning may have shifted over time. As indicated in Table 2.1, almost 
all chapters focus on the people theme of open world learning. This book brings 
together insights from 387,134 learners and educators in 136 unique learning con- 
texts, from short lab-exercises (Hillaire et al., 2022; Rets et al., 2022), in-class 
experiments (Anastasiou, 2022), online experiments (Korir, 2022), to longer inten- 
sive blended and online courses (Conde Gafaro, 2022; Vogiatzis et al., 2022) and 
MOOGCs (Chua, 2022; Rizvi et al., 2022). Many of the chapters do this mostly on 
a micro level, focussed on learners, teachers and technologies in one particular 
context or setting. This is important as a lot of macro studies have already been 
written on the affordances and limitations of open world learning. There is a dire 
need to conduct high-quality evidence-based research on the meso and micro 
level of learners and teachers of what works, and what does not, and why. 

The second important theme that will be discussed in this book is the practices 
in which learners and teachers find themselves. As context is everything in educa- 
tion, unpacking how practice influences learners and teachers to make sense of 
open world learning will help you, the reader, to use these insights for your own 


Table 2.1 Levels and 4Ps of open world learning discussed in this book 


Authors Chapter Level People Places Practices Properties 
Part | Learners and the power of language in an Open World 
Anastasiou 03 Digital stories in science: The role of story Micro 4 
sequencing 
Vogiatzis, Charitonos, 04 Can WhatsApp facilitate interaction? A case Micro v V 
Giaxoglou & Lewis study of adult language learning 
Conde Gaforo 05 First steps towards self-regulated learning: Micro v v 
Setting goals in MOOCs 
Chua 06 Discourse practices in MOOC discussions: A Micro v 
corpus linguistic approach 
Rets, Stickler, 07 Simplification of open educational resources in Micro "4 "4 
Coughlan & Astruc English: Its effect on text processing of English 
learners 
Rizvi, Rienties, 08 Culturally adaptive learning design: A mixed- Meso/ v V 4 
Rogaten & Kizilcec methods study of cross-cultural learning design micro 
preferences in MOOCs 
Part Il Innovative technologies in an Open World 
Iniesto, McAndrew, 09 Accessibility in MOOCs:The stakeholders' Meso/ "4 "4 "4 
Minocha & perspectives micro 
Coughlan 
Mohamud, Buckler, 10 Internet kiosks in Uganda: A window of Meso/ "4 "4 
Pitt & Twining opportunities? micro 
Korir, Slade, Holmes I| Eliciting students’ preferences for the use of Micro "4 
& Rienties their data for learning analytics: A 
crowdsourcing approach 
Hall, Herodotou & 12 Measuring player creativity in digital Micro v 
lacovides entertainment games using the creativity in 


gaming scale 
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Table 2.1 (Continued) 


Authors Chapter Level People Places Practices Properties 
Hillaire, Rienties, 13 Incorporating student opinion into opinion Micro v V 
Fenton-O'Creevy, mining: A student-sourced sentiment analysis 
Zdrahal & classifier 
Tempelaar 
Part Ill Educators and inclusive practice in an Open World 
Nguyen, Rienties & 14 Informing learning design in online education Meso v v y 
Whitelock using learning analytics of student engagement 
Iniesto & Hillaire 15 UDL and its implications in MOOC Meso V 
accessibility evaluation 
Srisontisuk 16 Practitioner's perspective on young children's Meso/ v v 
use of mobile technology micro 
Chaudhari, Littlejohn 17 Antecedents and consequences of uncertainties Meso/ "4 "4 
& Cross perceived by finance professionals micro 
Iwaniec-Thompson 18 The identity trajectories of older academics: Meso/ v v 
Workplace affordances and individual micro 


subjectivities 


Note: that all the reported studies included in this book went through formal Human Research Ethics Committee at the Open University UK, and 


received approval. 
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practice. Finally, places and properties will be explored in several Chapters. In 
Chapter 19 (Rienties, Hampel, Scanlon, & Whitelock, 2022b), we will bring 
together all these findings and insights and discuss how they have shifted our 
understanding of open world learning. 


Note 


1 https://iet.open.ac.uk/projects/owl 
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Chapter 3 


Digital stories in science 


The role of story sequencing 


Popi Anastasiou 


3.1 Introduction 


Stories, along with drawings and narratives, have been found to have a mediatory 
role in the construction of meaning (Bruner, 1990; Pantidos, 2017). They are a 
natural form of expression for people of any age and culture (Bruner, 1990). The 
technique of stories can be used in the organisation of events, facts, characters, and 
ideas into meaningful units (Hadzigeorgiou, 2016). A story takes its audience 
through a set of events, all the way from problem to solution and critical engage- 
ment with the solution (Polkinghorne, 1996), provoking active thinking and sup- 
porting meaning construction (Dettori & Paiva, 2009). 

Stories are frequently used in educational settings to support teaching and learn- 
ing (McEwan & Egan, 1995). As a teaching approach, using stories can stimulate 
students’ critical thinking skills (McDrury & Alterio, 2003) and help them to 
develop a variety of skills in communication, search, collaboration, and task com- 
pletion (Di Blas, Garzotto, Paolini, & Sabiescu, 2009). Educators "act as students' 
role models, agents of socialisation and brokers of knowledge" (Kucirkova, 2018, p. 
109), seeking to create and use suitable stories to convey content knowledge and 
to motivate students to learn (Jackson, 1995). Some educators may decide to create 
their own stories and then present them to students as a way of introducing new 
material (Robin, 2006). Others can have their students create their own stories 
(Robin, 2006). Furthermore, some may choose to co-create a story with their 
students, as an enjoyable process of negotiating a shared perspective (Kucirkova, 
2018). In contrast, as a learning approach the attention is on "finding meaningful 
ways for the students to make use stories related to their learning tasks, with the 
aim of facilitating and improving learning" (Dettori & Paiva, 2009, p. 56). 

A substantial number of changes in the presentation and delivery of stories is 
made possible by emerging technologies (Chen, Ferdig, & Wood, 2003). The 
advancement of technology, alongside the advent of relatively inexpensive (Davis, 
2004) digital tools, has shifted the focus from traditional to contemporary types of 
stories, digital stories. Digital tools enrich the process of creating and telling stories 
by providing new ways to support story authoring and fostering new forms of 
creativity (Di Blas et al., 2009), as also illustrated in Chapters 4 and 10 (Mohamud, 
Buckler, Pitt, & Twining, 2022; Vogiatzis et al., 2022). Enriching stories with digital 
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tools entail many advantages, such as more variation than traditional methods in 
current practice; personalisation of the learning experience; making explanation or 
the practicing of certain topics more compelling; facilitating the involvement of 
students in the process of learning (Gils, 2005). There is no doubt that the affor- 
dances offered by technology are valuable, yet the tools used for the creation of a 
digital story are transitory; thus, the focus should be on the learning process than 
the tools with which students engage to create a story. 

Sometimes, students are asked to produce a (fictional) story using written 
prompts, pictures, a wordless storybook, and videos (Pinto, Tarchi, & Bigozzi, 2018). 
Other times, students listen to a story and are asked to retell it at some later point 
(Pinto et al., 2018). Prompting students with a title, a picture, or not prompting 
them at all is a very common story-based technique that schoolteachers use. It is 
also a common research method to assess students’ narrative competence and task- 
comprehension by asking them to tell a story (Gazella & Stockman, 2003), based 
on a single prompt. 

Science stories differ from stories in the humanities in that they aim to improve 
the teaching and learning of science (Klassen, 2009). They are considered as a 
means of translating knowing into telling (Avraamidou & Osborne, 2009) and can 
prove useful in "communicating ideas and in making ideas coherent, memorable, 
and meaningful” (Millar & Osborne, 1998, p. 2013). However, it is often hard to 
accomplish the explanatory purpose in stories (Norris, Guilbert, Smith, Hakimelahi 
& Phillips, 2005), because of the need to use deductive-nomological explanations 
for scientific phenomena (Hadzigeorgiou, 2018). This chapter accepts the truth 
found in this limitation but stands by the view that science stories are not to be 
considered as a tool for testing concepts or explaining phenomena through experi- 
mentation and material evidence. 

The purpose of a science story is to describe natural phenomena and physical 
behaviours through a set of sequenced events to help its audience makes sense of 
what, how and why a phenomenon happens. Some researchers view as a limitation 
the use of science stories as descriptive explanations presented in a narrative form 
(through the use of anthropomorphism), because they are more suitable for young 
children (Hadzigeorgiou, 2018). This chapter embraces the suitability of science 
stories, because they are "attractive modes of communicating science" (Avraamidou 
& Osborne, 2009, p. 21) to younger audiences, helping them to understand the 
science content in an entertaining and engaging way (Klassen, 2009). 

Studies have sought to assess high-school students’ conceptual understanding of 
specific processes (e.g., natural selection) through narrative-based interventions 
(Prins, Avraamidou, & Goedhart, 2017). While their findings provided valuable 
insight about how narratives facilitated students’ understanding and engagement in 
a science lesson, their method focused on assessing students' performance after they 
were exposed to scientific information in narrative format, in comparison to other 
texts containing the same scientific factual information (Prins et al, 2017). 
Although it could occur that success or failure in a test situation accurately indi- 
cates cognitive development, it could also be inferred that such performance indi- 
cates nothing more than the student's ability to read the requirements of the test 
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(Adams, 2006). Thus, emphasis should be placed not only on the learning outcome 
(performance) but also on how students engage. Seeking to address this gap, this 
research placed emphasis on how students could engage in two different digital 
science-story activities. 

Studying digital stories in science learning is of interest because science stories 
have become an attractive mode of communicating science (Avraamidou & 
Osborne, 2009). The story perspective on science is based on the fact that scientific 
theories are fundamentally story-like, in the sense that they rely on metaphors, 
analogies, and conceptual frameworks (Hadzigeorgiou, 2018). Stories are a con- 
ceptual tool for providing coherence, continuity, and meaning to its contents 
(Hadzigeorgiou, 2016), which makes them a valuable instructional tool, especially 
in the context of science education, where abstract knowledge needs to be pre- 
sented in a way that makes sense to the students. 


3.2 Literature review and purpose 


The research documented in this chapter is concerned with how students engage 
collaboratively to prepare a digital story on a science subject they have already 
been taught. Collaborative learning settings encourage the construction of shared 
understanding through interaction with others, during which the participants are 
committed to or engaged in shared goals and problem solving (Dillenbourg, 
1999). 

A digital story can be presented in numerous ways. A story can be typed up as 
text, be performed, and videoed, be narrated and audio-recorded, be based on 
students' drawings and photographs, or combine various modes (Kucirkova, 2018). 
Authoring or editing a digital story does not necessarily require a new story. 
Students can author or edit an existing digital story by manipulating the presented 
information in it and determine its “order of presentation" (Hillmayr, Ziernwald, 
Reinhold, Hofer, & Reiss, 2020, p. 2). Langley (1995) defined order effects as dif- 
ferences in performance that arise from the same set of material being presented to 
learners in different orders. Order is a fundamental feature of all stories because the 
order in which story events are narrated relates to their order in thought (Montfort, 
2006). The order in which content is presented can "strongly influence what is 
learned ... and sometimes even whether the material is learned at all" (Ritter & 
Nerb, 2007, p. 3). There needs to be a strong relationship in the ordering of the 
content, because its presence among the story elements helps to determine the 
story's plot (Dettori & Paiva, 2009). The story elements do not have a life or mean- 
ing of their own; their "meaning is given by their place in the overall configuration 
of the sequence as a whole — its plot or fabula" (Bruner, 1990, p. 43). Thus, provid- 
ing students with an opportunity to interact with the story content and determine 
its order — contrary to other instruction methods that do not allow interaction — by 
controlling aspects of its presentation (Hillmayr et al., 2020) provokes active think- 
ing and supports meaning construction (Dettori & Paiva, 2009). 

In science education, digital stories are often assessed as a project on their own. 
Students are required to produce and develop a story based on pictures taken with 


32 Popi Anastasiou 


digital cameras or found online, add subtitles and background, and present their 
story to their classmates (Hung, Hwang, & Huang, 2012). Results from such 
projects seem to improve "learning motivation, attitude, problem-solving capa- 
bility and learning achievements of the students" (Hung et al., 2012, p. 376). 
Hung and his colleagues (2012) used an experimental study to provide valuable 
information about the use of digital storytelling in improving learning perfor- 
mance, but failed to evaluate how students might have engaged while working 
collaboratively. The current research sought to address that gap by taking an in- 
depth qualitative approach. 

Building on this literature, the purpose of this research was to encourage stu- 
dents to actively engage in a science topic through collaborative learning and 
digital story creation. The science topic was chosen given literature pointing to 
the fact that matter is among certain topics considered as problematic for teach- 
ers and students, who find them hard to teach and learn, accordingly (Clough et 
al., 2013). The problematic nature of these topics is to a large extent due to stu- 
dents’ misconceptions and alternative concepts about matter (Hadenfeldt, Liu, & 
Neumann, 2014), which are almost the same across different culture, ethnic 
groups, and class backgrounds (Gregory, 2009). Within the last decade, however, 
the focus of research has shifted towards investigating students’ learning progres- 
sions for the concept of matter (Hadenfeldt et al., 2014), meaning attention has 
been given to how students conceptualise matter, to what extent they are able to 
explain everyday phenomena, or how they develop an understanding of matter 
over time (Hadenfeldt et al., 2014). Taking everything into account, it seems that 
the relation between digital science stories and students! engagement in a prob- 
lematic topic, such as matter, is still underexplored. To this aim, this research 
sought to engage students in a difficult-to-learn science topic through a creative 
process that could lead to deep understanding of the topic of matter which has 
been identified as particularly problematic. The research questions that guided 
the research are as follows: 


1 How did students engage in each digital story activity? 
2 How did students perceive each activity? 


3.2.1 Research method and limitations 


This chapter utilised a qualitative approach and focused on the design and evalua- 
tion of two digital science story activities and aimed at informing teaching prac- 
tices. The two digital stories were the same in content but differed in presentation. 
Participants were two groups of primary students, aged 10—11 years old (Grade 5), 
who were taught the topic of matter two months before this research was con- 
ducted. For the purpose of the research, the researcher developed the two digital 
stories according to the curriculum material, while consulting with one of the 
Grade 5 teachers in order to make sure that the language and content were appro- 
priate for the students’ age-level. 
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3.2.2 Data collection procedure 


This research took place in a suburban public primary school in Athens, Greece. 
Two classes of students (middle-class, mostly white) aged 10—11 participated in 
this research. All students spoke Greek fluently. Class A consisted of 16 students 
and Class B of 15 students and all of them were described by their teachers as 
mixed-attainers, a statement that was also supported by students’ monthly test 
results in science and other subjects. For the purpose of data collection, students 
from each class were grouped into small teams of three to four students and 
worked collaboratively. The two classes completed two different digital story 
activities on different days, during school hours. The first activity (named story 
sequencing) included a digital story about matter, broken down into fifteen scenes, 
which were presented to students in a random order. Students had to first deter- 
mine the plot by ordering the story scenes, and then narrate it by making written 
commentaries. The second story (named story verbalisation) about matter was 
presented to students in its pre-defined order, as a complete story, and students 
had to narrate it by verbalising its content through written commentaries. Class A 
completed the story sequencing activity and Class B the story verbalisation activ- 
ity. Neither activity included any audio or visual commentaries so that students 
could make their own narration 

The content of the stories was based on the curriculum material for the teach- 
ing of matter in Grade 5 and was the same in both activities. The ordering of the 
content in the story verbalisation activity was similar to the teaching sequence for 
matter taught at school, according to the Greek programme of study for science for 
Grade 5 (Figure 3.1). The proposed teaching sequence followed a hierarchical pre- 
sentation of concepts, introducing first the simple concepts before moving on to 
the more complex concepts of which they are part. It started with an introduction 
to the three states of matter and then continued with the changes of conditions, 
such as melting and freezing, and evaporation and condensation. It ended with the 
concepts of heat and temperature. 
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Figure 3.1 The content of the story and its order of presentation in the story 
verbalisation activity. 
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Table 3.1 Data collected in response to the two research questions 


Data collected Class A Class B Research questions 
Resultant Randomly Pre-defined | How do students 
stories ordered scenes order of engage in each 

(story scenes (story digital story 
sequencing) verbalisation) activity? 
Group 2 teams 2 teams 2 How do students 
interviews perceive each 
activity? 


The role of the researcher was to participate as a silent observer, keeping notes 
about the nature of students’ interaction. With the students’ consent, there were 
also audio-recorders to record each team’s interaction. The Class A teacher was 
present as an observer and alongside the researcher offered practical and technical 
guidance where necessary. 

After the two digital story activities were implemented, follow-up group inter- 
views were conducted with a subset of teams of students from each class. The 
interviews with the students aimed at collecting information about students’ per- 
ception of the conceptual nature of each digital story activity. The students were 
selected in such a way to obtain a representative group in terms of their abilities 
(medium attainers and low attainers), for which information was provided by their 
teachers. The research data for this research consisted of the students’ resultant digi- 
tal stories and group interviews, as outlined in Table 3.1. 


3.2.3 Data analysis 


Seeking to evaluate how students might have engaged through each activity, stu- 
dents' story elements were analysed according to the revised framework of Norris 
et al’s (2005) framework for science narratives (Table 3.2). The group interviews 
were analysed using the hybrid process of thematic analysis (Boyatzis, 1998; Fereday 
& Muir-Cochrane, 2006). The hybrid process of combining inductive and deduc- 
tive thematic analysis involved all the steps of the data-driven approach and at the 
same time allowed the researcher to use pre-conceived theories as a guide for 
articulating meaningful themes (Boyatzis, 1998). In the case of this research, the 
analysis started as data-driven, looking for patterns and themes in the data (Braun 
and Clarke, 2006). Attention was paid to basic concepts that recurred and parts of 
the data that described similar concepts were color-coded. Then the hybrid process 
progressed to theory-driven coding and analysis, drawing on codes from the revised 
science story framework of Norris et al. (2005). The compare-and-contrast phases 
of this process helped to minimise possible distortions (Boyatzis, 1998) and to over- 
come the possibility of the researcher biases in the qualitative analysis of one activ- 
ity against the other. 
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Table 3.2 Key elements of a science story (adapted from Norris et al., 2005) 


Story Element Description 

Purpose The science story helps to understand the natural world 
and people's place in it. 

Events The science story events are chronologically related/ 
sequenced, in an explicit or implied way. 

Structure The science stories are structured around two independent 


time sequences—the sequence of plot events and the 
sequence in which the events are related. 

Agency The science stories involve human beings or other moral 
agents who cause and experience events and are 
responsible for their actions. 

Narrator The narrator determines the point and purpose of the 
narrative and selects the events and their sequence. The 
narrator fashions a sequence of events into a significant 
whole. 


3.2.4 Presentation of findings 


The analysis of findings according to the science framework (Table 3.1) revealed 
that the two activities had a different conceptual nature. From the five essential 
elements, story sequencing (Class A) and story verbalisation (Class B) shared pur- 
pose and agency, while they differed in events, structure, and narrator (Table 3.3). Both 
classes provided scientific explanations about matter, which was the purpose of the 
science stories. Also, both classes involved agency in their science stories by attribut- 
ing human-behaving or material behaviours to the story-characters. Where the 
two classes differed was in how they structured the story events, with Class A being 
free to sequence the story events, determining their chronological order and causal 
relation. In contrast, Class B followed the pre-defined chronological and causal 
sequence of events that were based on the proposed teaching sequencing for mat- 
ter. Another key difference was in the element of structure: Class A’s activity enabled 
them to sequence the story events based on how they related to each other, as 
opposed to Class B’s activity that involved a sequence of plot events. Lastly, the role 
of the narrator was distinctively different through the two activities. Class A were 
free to determine the story plot by selecting the events that related and deciding 
about their sequence, whereas Class B determined the story plot by verbalising a 
pre-defined sequence of events. 

In order to visualise the differences found in the story elements of the two 
activities, Figure 3.2 and Figure 3.3 illustrate the artefacts from two teams (Team 
A/Class A) and (Team B/Class B) that provide practical evidence of the conceptual 
nature of each activity. Regarding the element of events, Team A students deter- 
mined the sequence of the scenes starting with scene 4, then scenes 3, 8, 14, 2, 1,9, 
10, 6, 7, 15, 11, 13, 12 and ending with scene 5. With reference to the element of 
structure, students followed a mixed ordering of the scenes. They started with the 
introductory concepts (scenes 4,3), moved to the changes of conditions (scenes 
8,14), back to the introductory (scenes 2,1), to changes (scenes 9,10) again, then 
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Table 3.3 Analysis of students' resultant digital stories based on the five key 
elements for science stories 


Key elements of 
a science story 


Story sequencing activity 


(Class A) 


Story verbalisation activity 


(Class B) 


Purpose States of matter States of matter 
Events Free-choice chronological Pre-defined chronological 
and causal sequence of and causal sequence of 
events events 
Structure Sequence of related events Sequence of plot events 
Agency Humanlike agents and other  Humanlike agents and other 
entities entities 
Narrator Determines the story plot Determines the story plot by 
by selecting the events verbalising the pre-defined 
and their sequence events and their sequence 
$ o’ S 27. I "Y 
B3. es 
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Liquid is a natural state of 
matter. 

Liquids flow as their 
particles move around 


- Oh, the frozen lemonade 
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temperature room 
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low temperature room, 


Mountains are snowy and 
cold. 

When the sun appeared, 
ice melted,and streams 


Solid is a natural state of 
matter. 

Its particles move slightly 
while keeping their 
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each other. 
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-Hi everyone! | play 
football very well! 

-I can also make a 
sculpture of myself as a 
king! 


-Oh, no! My ice-cream is 
melting and the water 
with ice-cubes got really 
hot. This process is called 
MELTING. 

-Brrrr, it's so cold, 0°C 
and | am freezing! This 


-Aaah, finally | ‘m out! 
Fresh air! 

-Oh dear! I’ve never seen 
so much cold before! 


Gas is a natural state of 
matter. 

In gases, particles move 
around by constantly 
changing their posititions. 


Gases can occupy all 
space in these jars. 


process is called 
FREEZING. 


Scene 15 


Scene 11 


Scene 13 


Scene 12 


Scene 5 


Water evaporates due to 
the sun's heat. 


Solidy goes into Greeny's 
bathroom and changes 
the room's temperature. 
First he turns it up and 
then he turns it down. 


Particles are, first, turning 
hot and then turning cold. 


Solidy swept the window 
with his hand because 
some gas had been 
condensed and turned 
into liquid. 

Then, an angry Greeny 
appeared but when he 
got closer to the window, 
he almost dropped his 
towel ... and so Solidy run 
away. 


Gasy drank a liquid that 
made him burp ... so he 
scared Greeny, who 
dropped his liquid onto 
Solidy, who tripped over 
his sculpture. 


Figure 3.2 Team A/Class A digital story (story sequencing activity). 
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introductory (scenes 6,7), then back to changes (15,11, 13, 12), and finished with 
an introductory concept (scene 5). Considering the element of the narrator, Team 


A engaged in a process of manipulating the learning content to match students’ 


preferred presentation. 
The story verbalisation activity, by contrast, involved a pre-defined story, with all 


its pieces together, and students had to narrate its plot by verbalising its existing 


context, as illustrated in Figure 3.3. The story verbalisation activity did not allow 


students to make their own decisions about the ordering of events or the structure of 


the story. Team B had to follow a pre-defined structure of events, starting with the 
introduction of the three states of matter in scenes 1—2, 3-4, and 5-7. Then, they 
continued with the changes of conditions, such as melting and freezing in scenes 


8-10 and evaporation and condensation in scenes 11-12. They finished with heat 


and temperature in scenes 13—15. As such, the role of the narrator was that of deter- 


mining the story plot by verbalising the pre-defined content (from scenes 1, 2, 3, 4, 
5,6,7,8,9,10,11, 12, 13, 14 to 15). That is, through the story verbalisation activity 
students engaged in a process of verbally producing an existing content, presented 


to them as a complete story. Like with the story sequencing activity, students' 
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-Argh, | can’t see! 
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stomach...Brrrrrr!! 

-Argh, my sculpture! | am 
going to ‘kill’ you! 
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Scene 10 


In gases, particles move 
away from each other as 
far as possible. 


Fire's particles are gases 
that spread out in a 
container, taking thus its 
shape. 


-Mmm, my ice cream is so 
delicious! Luckily, l've got 
my ice cubes in case | get 
hot. Can’t wait to go inside. 
-Let’s go! 


-It's too hot in here! Both 
my ice-cubes and ice- 
cream melt! This 
suggests that melting 
began. 

-Brrr, it's too cold in 
here.. my water turned 
into an ice-cube.This 
means that freezing 
started. 


-Taram! Ah, finally! 

-It was too hot! | melted! 
-I froze! 

-Here you go! 

-Thanks! 
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Scene 11 
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Scene 12 


Scene 13 


| Scene 15 


-Hmm, | am going in. 
-Lalala! Oh no, water 
evaporated! 

-Let's see here! A ha! 


-Hey, you, come here! 
-Oops, that was close! 


The thermometer is at 63 
degress Celsius. 

When an object gets hot, 
particles move faster. 
When it cools down, 
particles move slower. 


When the sun is burning, 
it can make water 
levaporate in the 
mountains. The initial 
temperature is 0°C. 


The sun is too intense 
and water in the yard 
evaporates ... and from 
liquid turns into gas. 


Figure 3.3 Team B/Class B digital story (story verbalisation activity). 
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resultant stories from the story verbalisation activity were not compared to the 
original story, thus there was not a right or wrong plot. 

Moreover, the analysis of the group interviews (Table 3.4) resulted in character- 
ising students' perceptions of each activity's conceptual nature. These are presented 
below in a series of main assertions as those became evident in the data analysis: (a) 
some students felt that they owned the story sequencing activity more than the 
story verbalisation activity, (b) most students thought that each activity was enjoy- 
able and exciting, (c) some students felt that story verbalisation was not as hard as 
story sequencing, and (d) some students were confused about the conceptual 
nature of story sequencing. 

The first theme, ownership of creation, revealed that students liked the version of 
activity on which they worked, and felt they owned it in their way. Class A from 
the story sequencing activity liked the fact that they were free to order the scenes 
according to their own understanding and make commentaries about them. 
However, some students found this level of ownership hard. Unlike Class A stu- 
dents, Class B students from the story verbalisation activity were more excited 
about the way they owned their activity. 

The second theme, fun, indicated that both Class A and Class B students enjoyed 
their activities, and they would like to do them again in other lessons. Class A stu- 
dents said that they had never before worked through similar activities. Class B 
students found it fun and expressed a preference over the story sequencing activity 
because story verbalisation was an easier activity. 

Finally, the third theme, conceptual complexity revealed the complexity of the story 
sequencing activity’s conceptual nature, which sometimes tired and confused stu- 
dents. The absence of visual or audio narration and the presentation of the scenes 
in a random order seemed to have troubled students. In contrast, students from the 
story verbalisation activity did not face such difficulties. As findings revealed, some 
participants showed a preference for easy tasks that did not require much effort. 
Further evidence from the group interviews' analysis showed that while both Class 
A and Class B students enjoyed their activity, the story sequencing activity as 
opposed to the story verbalisation one required harder thinking and more concen- 
tration, which tired some Class A students. 


3.3 Discussion and moving forward 


Chapter 3 explored how students might have engaged in collaborative learning 
and digital story creation in a science class. The story sequencing activity enabled 
students to actively engage with the learning content and to manipulate — to a 
certain extent — the presented information, determining their preferred order of 
their story. Interacting with the story content allowed Class A students to make 
their own decisions about the ordering and sequencing of the events, based on 
their understanding of matter. This process positioned Class A students as makers 
of the story, and involved physical and mental intensity, attention to detail and 
reflection. The physicality of manipulating and methodically arranging the story 
events so that they related and made sense, pushed students to think 


Table 3.4 Open coding for students' perceptions of each activity 
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not only harder but also differently, which is in itself a reflective meaning-making 
endeavour (Matthews-DeNatale, 2013). 

The process of story making encouraged students to make decisions, think inde- 
pendently, and seek answers (Kucirkova, 2019). Decision-making as an explanation- 
based process, pushed students as decision makers to make sense of the available 
information to aid the selection process (Jonassen, 2011). Through story sequenc- 
ing, students were required to make a choice among alternative claims and gener- 
ate arguments to justify each option. It is through the process of debate, discussion, 
and reflection that students engage in the learning process, because they work 
together to storyboard and edit their digital stories (Standley, 2003). Students’ 
decision-making depended on how they argued for and against each option on the 
basis of their existing knowledge and how they combined those arguments to 
reach a joint decision (Jonassen, 2011). Their final decisions resulted from the pro- 
cess of supporting or rejecting alternative claims/decisions and were constructed as 
an explanatory representation in story form that contained causal accounts of the 
evidence (Jonassen, 2011). 

Positioning students as story makers not only enabled them to control aspects of 
the story presentation (Moreno & Mayer, 2007), and thus to take ownership of 
their creation, but also challenged their existing assumptions (Kucirkova, 2019). 
Engaging in the process of making a story depends on how one can see their 
understanding of something come together and make sense. The sharing of ideas 
and information allowed students to make meaning based on their collaborative 
interaction. The findings indicated that students strategically related scenes together, 
when they tried to conceptualise matter by matching together chunks of informa- 
tion based on their shared understanding and existing knowledge. 

By contrast, findings from the story verbalisation activity pointed out the 
limited options that Class B students had when asked to determine the plot of 
a pre-defined story. Having to verbalise the content of the story, students did 
not have to reflect on either purpose or strategy in linking the science con- 
cepts between them. In particular, the story verbalisation activity allowed 
students to think about the content of the story, not its presentation/order, so 
students had to rely on retrieving and recalling prior knowledge. This process 
positioned Class B students as tellers of an existing story, which did not require 
any active engagement with the ordering of the content. Thus, the strategies of 
implementation that students followed in verbalising an existing story by 
determining its plot involved the effort to relate the story parts to previous 
knowledge. In doing so, Class B students might have tried to recall (parts of) 
content in the specific teaching sequence they were taught it. Such a sequence 
is dependent on teachers’ and curriculum-developers’ interpretation of how 
specific knowledge should be taught, what certain concepts should precede 
before others because without them any subsequent knowledge would not be 
comprehended. 

Generally, both digital science-story activities involved a creative process that 
enabled students to determine either the story content and its plot, or its plot. 
Each activity had a different level of conceptual complexity involved. Taking 
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also into consideration participants’ attainment level and previous knowledge in 
the specific topic, it was expected for some students from both classes to face 
difficulties in engaging with each activity. To conclude, the process of making a 
story as opposed to telling a story emphasised students’ agency in making 
choices, thinking independently and seeking answers through shared under- 
standing. The argument would be that engaging students in harder versions of 
the same task could generate better results (Brown, Roediger, & McDaniel, 
2014), as students might learn more in the least preferred conditions (Kelly & 
Tangney, 2006). 


3.3.1 Implications for practice 


Positioning students as digital story makers gives them the opportunity to take 
responsibility for every step of the creation process while determining the presen- 
tation of the content and negotiating a shared understanding of ideas and informa- 
tion. Whilst students may find the story sequencing activity as conceptually difficult 
and at times tiring, it could be implied that challenging and hard activities deepen 
engagement and promote learning by doing, which is a meaning-making approach 
in itself. Activities that follow the proposed teaching sequence are not always ben- 
eficial in engaging students in science learning. The proposed teaching sequence 
depends on a hierarchical interpretation of knowledge, meaning that certain simple 
concepts need to be learned before other more complex of which they are part. It 
is often the case that the fragmented presentation of knowledge through a hierar- 
chical teaching sequence can cause barriers to students’ understanding of difficult- 
to-learn science topics. Therefore, using digital stories to communicate scientific 
information is valuable and should be further explored to investigate its potential 
for science teaching and learning. 
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Chapter 4 


Can WhatsApp facilitate 
interaction? A case study of adult 
language learning 


Dimitrios Vogiatzis, Koula Charitonos, Korina Giaxoglou 
and Tim Lewis 


4.1 Introduction 


The ubiquity of technology and the proliferation of digital technologies in our 
everyday life have transformed the landscape of education. Technological develop- 
ments in the 21st century provide opportunities for inclusion and access to a wide 
range of information, knowledge, and learning. Consequently, today, technology 
significantly contributes to educational environments in which emerging digital 
tools are used to facilitate teaching and learning in several ways. The advent of 
Mobile Instant Messaging application (MIM) like WhatsApp has influenced the 
ways in which people communicate in everyday life and caught scholarly attention 
in exploring their potential in language learning settings. 

Chapter 4 presents an exploratory case study undertaken in an adult education 
centre and aims to examine the extent to which WhatsApp can be used as a means 
of facilitating learner-to-learner and teacher-to-learner interactions. In line with 
the purpose of this book, Chapter 4 explores the practical applications of WhatsApp 
and aims to identify the “enablers” and “disablers” of this MIM app for language 
learning and interaction. 


4.1.1 Literature review 


The advent of MIM applications like WhatsApp has shaped the communicative 
practices of people around the globe, as also indicated in Chapter 16 (Srisontisuk, 
2022). These mobile communication services differ from traditional SMS, since 
they enable users to send and receive text messages, images, video, audio, and loca- 
tion information in real-time to individuals or groups of friends at no cost (Church 
& de Oliveira, 2013). WhatsApp is an internet-based cross-platform instant mes- 
saging application for smartphones (also accessible via desktop computers, laptops, 
and tablets). To date, WhatsApp is the most popular MIM application with two 
billion monthly active users worldwide (Statista, 2020). 
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4.1.2 WhatsApp as a language learning environment 


The popularity growth of WhatsApp and its features has led many scholars to 
explore its potential in language learning settings. Specifically, this MIM app was 
used to develop language learners’ writing (Ahmed, 2019) and reading skills (Alzubi 
& Singh, 2018), as well as their vocabulary acquisition (Lai, 2016) and listening 
skills (Fauzi & Angkasawati, 2019). A recent systematic literature review conducted 
by Kartal (2019) explored the effectiveness of WhatsApp and identified 37 studies 
in the field of language learning and teaching. The review concluded that WhatsApp 
is a useful tool that can facilitate language learning, but highlighted that the studies 
examined did not clearly discuss the theoretical underpinnings of WhatsApp use. 
Moreover, according to Kartal (2019), some of the articles analysed provided nei- 
ther any information about the levels of participation in the online environment 
nor any details about the implementation process. 

In terms of methods, the studies examined relied heavily on quantitative data 
(questionnaires and pre- and post-tests), while only two studies used observation 
techniques to investigate participants’ actual use of WhatsApp for language learn- 
ing. This shows that there is a lack of measurements such as observation of partici- 
pants’ online activities and analyses of their sharing practices. Empirical frameworks 
for the analysis of sharing practices in Social Network Sites (SNS), as proposed by 
Androutsopoulos (2014), can capture learners’ actual participation in these online 
environments (see Section 4.3.3). 

As regards the educational settings in which WhatsApp has been implemented, 
the review showed that most of the studies (65%) were conducted in higher educa- 
tion contexts, while none of them examined the effects of WhatsApp in adult 
education. Adult and community learning is a different environment from higher 
education in important respects. Learners are more mature (often of retirement 
age), and they are studying purely for personal interest, rather than formal qualifi- 
cations. Consequently, their learning is not assessed or graded, while they are much 
less exposed to the target language than they would be at university. 


4.1.3 WhatsApp and language interaction 


Interaction is considered one of the main mechanisms by which languages are 
learnt (Mackey et al., 2012), as also indicated in Chapter 5 (Conde Gafaro, 2022) 
and Chapter 6 (Chua, 2022). Specifically, Aburezeq and Ishtaiwa (2013) examined 
the impact of WhatsApp on interaction in an Arabic language teaching course. The 
findings indicated that most participants (71%) perceived that WhatsApp had the 
power to enhance peer-to-peer interactions. However, this study relied on inter- 
view data and did not provide any empirical evidence on how WhatsApp can 
facilitate such interactions. As regards student-to-student interactions, Keogh 
(2017) argued that WhatsApp can increase learners’ involvement and accommo- 
date various traits of communities of practice (1.e., co-construction of knowledge, 
scaffolding, etc.). Nevertheless, this study revealed that there was “a lack of true 
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dialogue" when learners interacted with each other, while most of the discussion 
on WhatsApp were teacher-initiated (Keogh, 2017, p. 102). 

Another study examined the benefits of WhatsApp to develop second language 
wriüng (Andujar, 2016). Findings indicate that WhatsApp contributed to the 
development of accuracy in second language writing. As regards interaction, 
Andujar (2016, p. 63) argued that “WhatsApp constitutes a powerful educational 
tool to encourage second language interaction among participants" and empha- 
sised the need for further research mentioning that “[i]ts tremendous potential to 
activate students' involvement remains one of the least exploited functionalities of 
mobile phones". 


4.1.4 Aims of the study and research questions 


In line with the literature discussed above, Chapter 4 places attention on the 
opportunities that this MIM application can provide in terms of learner-to-learner 
and teacher-to-learner interaction (including peer-to-peer feedback interaction). 
Specifically, the study presented in Chapter 4 seeks to address the following research 
questions: 


1 How do adult learners and their teacher participate and contribute while 
using WhatsApp for language learning purposes? 

2 To what extent can the use of WhatsApp facilitate (or impede) language 
learning interaction in an adult education context? 


4.2 Research context and methods 


This study explored the use of WhatsApp by a group of adult learners of German 
language in an adult education centre in the UK. In this context, learners attended 
classes once a week (90 minutes to two hours), running for three terms of ten 
weeks each (September-December, January-April, April—July, 2019-2020). Adult 
learners in this school typically attended 30 lessons a year (2 hours a week) and 
were exposed to the target language only intermittently, while the long breaks 
between the terms made it difficult for them to recall and assimilate knowledge. 
WhatsApp use in this study aimed to alleviate these challenges by extending the 
limited class time and providing a friendly online environment that could facilitate 
learner-to-learner and teacher-to-learner interactions. 


4.2.1 Choice of technology and implementation process 


The selection of the MIM was made by considering teachers and learners’ needs 
and preferences, as well as their familiarity with the various MIMs available today. 
A WhatsApp group was created by the teacher, and learners were invited to join 
the group. The first author was also added as member of the group with the sole 
aim of observing and recording interactions amongst the members of the group 
without disturbing the naturally occurring exchanges. Given that one of the aims 
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was to examine how the teacher and learners adapted to WhatsApp and impul- 
sively used it to facilitate their language teaching and learning, respectively, no 
formal rules were set by the researcher. Participants were given complete freedom 
to interact, initiate conversation, and share any information related to their German 
language learning. 


4.2.2 Participants’ profiles 


Participants were eight native speakers of English who aimed to learn German as 
a foreign language purely for personal interest. All learners were adults and fell into 
a range of age categories, while two were in the age range of 40—49. As identified 
in Chapter 19 (Iwaniec-Thompson, 2022), six learners were in the older category, 
two in 50-59, respectively, three in 60—69 years old, and one participant being over 
70 years old. 


4.2.3 Data collection and analysis 


The research reported in Chapter 4 is part of a broader PhD thesis that followed a 
mixed-methods approach, employing pre- and post-questionnaires, combined 
with online observations and interviews. In Chapter 4, we explore the ways in 
which online interactions took place in the WhatsApp group by examining the 
data derived from the online observations and the semi-structured interviews. 

Throughout the duration of the study (n = 22 weeks), systematic online obser- 
vations were performed, and all participants’ posts and comments were archived 
and analysed to develop an in-depth understanding of informants’ online partici- 
pation. Androutsopoulos’ (2014) empirical framework for the analysis of sharing 
practices in SNSs was adapted and used for the purposes of this study. The analysis 
was carried out in three stages. 

First, a quantitative analysis of participants’ online contributions was carried out. 
Aiming to examine the extent to which the learners and the teacher participated 
and contributed to the WhatsApp group, all chat entries were counted, as also 
described in Chapter 10 (Chua, 2022). A chat entry was identified when a partici- 
pant published a message in the text-chat window (Cho, 2017), including typed 
text, embedded images, videos, You Tube videos, emojis, links to web content, or a 
combination thereof. Counting and coding the chat entries in this online environ- 
ment distinguishes between initiating and responding contributions (Androutso- 
poulos, 2014). An initial entry was identified when a participant published a message 
to initiate conversation on a new topic, while a chat entry was categorised as a 
responsive contribution when a participant replied to an initial entry within the 
online environment. 

'The second step involved the identification and selection of relevant communi- 
cative events (or "significant moments", the term used in Androutsopoulos, 2014), 
which were then qualitatively analysed. As proposed in Androutsopoulos (2015) 
the basic unit of analysis is not a single post but a communicative event which is 
defined as a “spatially and temporally delimited, multi-authored sequence of 
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contributions" (Androutsopoulos, 2014, p. 7) and consists of an initial chat entry 
followed by other users' responsive contributions. To identify and select communi- 
cative events relevant to the analysis, three criteria were followed, namely (1) “rep- 
etition", (2) "responsiveness", and (3) "reflexivity" (see Androutsopoulos, 2014, 
p. 8). More specifically, repetition involved those communicative events in which 
participants repetitively used similarly styled chat entries. Events in terms of 
responsiveness were identified when posts receive a significant number of responses 
by the networked audience. Finally, reflexivity included communicative events 
where participants self-reflect upon their sharing practices. Such reflections are 
“elicited in secondary data sources such as interviews” which can “offer important 
pointers back to acts of sharing in the digital data” (Androutsopoulos, 2014, p. 8). 

The third stage involved a qualitative analysis of the selected communicative 
events aiming to provide an empirically evidenced account of how WhatsApp was 
used as means of facilitating participants’ language learning in this adult education 
setting. As proposed in Androutsopoulos (2014), this qualitative analysis was per- 
formed on three different levels, namely, "selecting", "styling", and “negotiating”. 
Selecting concerns what participants chose to share and why, while styling involved 
the entextualisation of what was shared (i.e., how participants style their content), 
and negotiating deals with the audience engagement (i.e., how participants negoti- 
ate what is shared with other users) (see Androutsopoulos, 2014, p. 8). It is worth 
noting that "sharing" in SNSs is conceptualised as a mode of participation that 
incorporates both communication (where sharing is "telling") and distribution 
(i.e., sharing digital content) (see John, 2013). 

Finally, after the completion of the study, semi-structured interviews (n=6) were 
conducted to provide more insight into the results derived from the online data 
and to identify the reasons why learners demonstrated specific participation pat- 
terns in the online environment. The interview data were subjected to thematic 
analysis following the six steps proposed by Braun and Clarke (2006). 


4.3 Results 
4.3.1 Participation and sharing practices in WhatsApp 


As Table 4.1 shows, the learners and their teachers made a total of 243 chat entries. 
Approximately one-third of chat entries (n=78, 32.1%) were made by the teacher, 
emphasising that his contribution in initiating, coordinating, and participating in 
other interactions was key. Apart from the individual messages, the teacher made 22 
(56.4%) initial entries and replied to 56 (27%) of the posts. These findings suggest 
that the teacher had a central role in the WhatsApp group. 

Moreover, as Table 4.1 illustrates levels of participation among the learners var- 
ied. One learner, Emma, was very active in the online environment posting 52 out 
of 243 chat entries (21.4%), while others like Petra and Klaus were more passive 
posting six and only one chat entry, respectively. As regards learners’ contribution, 
the data made clear that their participation in the WhatsApp group was reactive, 
rather than proactive. Specifically, learners rarely took the initiative to start a new 
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Table 4.1 Level of participation and contribution 


Teacher Emma Louisa Otto Frieda Jürgen Helmuth Petra Klaus Total 


Individual 78 52 27 23 20 19 17 6 l 243 
chat 
entries 

Initial 22 9 l 2 3 2 0 0 0 39 
Entries 

Replies 56 43 26 21 17 17 17 6 l 204 


topic as the number of initial entries per learner was very low, while most of their 
contributions (89.7%) were replies to previous conversations or activities posted by 
the teacher. The reactive nature of learners’ participation was obvious from the 
early stages of the intervention. A possible reason that elucidates learners’ reactive 
participation emerged from the analysis of the interview data. Specifically, all learn- 
ers felt that they did not “have the right or the authority to initiate something so 
we [they] sit back wait for [name of the teacher] and then we respond” (Louisa). 
The view that only the teacher had the authority to initiate conversation in the 
online environment points out the different dynamics of participating and sharing 
when using WhatsApp for language learning purposes. It seems that this was a rule 
set by learners arbitrarily, as it was never formally set. Instead, learners were given 
complete freedom to interact, initiate conversations and share any information 
they wanted on WhatsApp, but opted for reproducing the participation dynamics 
of classroom interaction in this digital context. 


4.3.2 Activities in the WhatsApp group 


A total of twelve language learning activities were uploaded by the teacher. Half of 
them were writing tasks that required the learners to compose sentences using 
vocabulary and/or verbs supplied by the teacher, while three activities asked learn- 
ers to describe their weekend using the target language. The teacher also uploaded 
another activity asking the learners to plan a trip to Germany discussing its practi- 
calities (where to go, what to do, what to see, when to go and for how long), while 
another activity required learners to compose a dialogue in a restaurant or a shop. 
Finally, an activity shared by the German teacher asked learners to choose and 
describe a city in Germany or Austria for their peers to guess which it was. 

Three of the activities presented above were chosen and qualitatively analysed 
following the selection criteria (i.e., repetition, responsiveness, and reflexivity) as 
discussed in Section 4.2.3 Specifically, the first communicative event satisfied the 
first criterion (i.e., repetition) and offered a representative example of the activities 
which were repetitively used by the teacher. The second event (see Section 4.3.4) 
was selected because it received the most significant number of responses by the 
learners (i.e., responsiveness), while the third communicative event (see Section 
4.3.5) entailed participants’ self-reflection upon their sharing practices in the online 
environment (i.e., reflexivity). 
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4.3.3 Example |: “Constructing a sentence using the verb 
zutreffen auf" 


The first communicative event offers a representative example of how the teacher 
used the messaging application to distribute writing tasks. As Figure 4.1 illustrates, 
the teacher posted a writing task asking the learners to construct a sentence using 
the separable verb zutreffen auf (apply to). Three learners, Louisa, Frieda and Emma, 
followed the instructions provided by the teacher and generated sentences using 
the suggested verb. The teacher did not share any follow-up comments, possibly 
waiting for the rest of the learners to provide their answers to the writing task. 
After three days, he posted a text entry thanking the learners who participated in 
this activity and affirming that they understood the given verb well. His chat entry 
marked the end of this communicative event. 


15 MARCH 2019 


Teacher 

Kann jemand das trennbare Verb 
‘zutreffen auf’ in einem Satz 
benutzen? 10:51 


Louisa 


GA 


(&) Vielleicht..? Die neue 
Regulierung wird auf alle Bürger 


Teacher: Can anyone use the 
separable verb 'apply to' in a 
sentence? 


Louisa: (Emoji) Maybe ...? The 
new regulation will apply to all 
citizens. 


zutreffen. 11:43 

Frieda Frieda: What happened to us then 
Was hat auf uns dann zugetroffen, applies to you. 

trifft nun auf Ihnen zu. 16:03 


Die Einwande der Britische Regierung 
gegen den sogenannten Backstop, 
dass der Backstop teilt Nordirland 
von dem Rest Großbritanniens, 
zutrefft genau auf der 
vorgeschlagenen Zolltarife, die die 
Britische Regierung kündigte an, die 
zutreffen auf im Fall einen 
No-Deal-Brexit. & 


18 MARCH 2019 


Teacher 

Vielen Dank , und ; Ihr 

habt das Wort gut verstanden! "- 
17:38 


« 


Emma: The British government's 
objections to the so-called 
backstop that the backstop 
divides Northern Ireland from the 
rest of the United Kingdom are 
accurate on the proposed tariffs 
that the British government 
announced that it would apply in 
the event of a no-deal Brexit. 
(Emoji) 


Teacher: Thank you (Louisa), 
(Frieda) and (Emma). You 
understood the word well! 
(Emoji) 


Figure 4.1 Example of a writing task posted by the teacher. 


Teacher 


A new activity this week.. Choose a 
destination in Germany or Austria 
but don't tell the rest of the group. 
Imagine you're going to travel there. 
Write to the group describing its 
geographical location, how you will 
travel, what is special about this 
place and why you suggested it. The 
rest have to guess where it is. 

Here goes... 

Die Stadt liegt im Norden 
Deutschlands. Ich werde dorthin 
fliegen. Die Stadt ist zugleich Stadt 
und Bundesland. Ich habe diese Stadt 
gewáhlt, weil ich als Student da war, 


und immer nicht Freunde dort habe. 
Wo werde ich sein? (©) Tid 


Helmuth 


Es ist Hannover? 11:53 


Frieda 

Ich glaube, dass die beiden 

von Hamburg und Bremen 
Bundeslande sind, und also ist sie 
eine dieser Stadte? 12:10 


Helmuth 

Ich bin beantworten falsch, aber 
Hannover war ein Stadt und 
Bundeslande für 92 Tages einmal (::) 


12:27 


Frieda 


12:54 


Emma 

Diese Stadt kónnte entweder 
Hamburg oder Bremen sein. Ich 
glaube, dass Bremen keinen großen 
Flughafen hat. Daher wáhle ich 


a 1309 


Figure 4.1 (Continued) 
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Teacher: 


A new activity this week... Choose a 
destination in Germany or Austria but 
don’t tell the rest of the group. Imagine 
you're going to travel there. Write to the 
group describing its geographical 
location, how you will travel, what is 
special about this place and why you 
suggested it. The rest have to guess where 
itis. Here goes... 


The city is in the north of Germany. I will 
fly there. The city is both a city and a 
federal state. I chose this city because I 
was there as a student and always have 
friends there. Where will I be? (emoji) 


Helmuth: 


Is it Hanover? 


Frieda: 


Ithink that Hamburg and Bremen are 
federal states, so is it one of those cities? 


Helmuth: 


I'm answering wrong, but Hanover was a 
city and a state for 92 days once (emoji) 


Frieda: 


(emoji) 


Emma: 


This city could be either Hamburg or 
Bremen. I don't think Bremen has a big 
airport. That's why I choose Hamburg 
(emoji) 


5l 
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Teacher Teacher: 
Keiner hat bisher richtig No one has answered correctly so far...! 
geantwortet...) i (emoji) 
Frieda Frieda: 
Es muss Berlin sein - die dritte Stadt, | It must be Berlin -the third city that is 
die auch ein Bundesland ist? 17::| also a federal state? 
Jürgen Jürgen: 
iellei eo Maybe Bremen? (emojis 
Vielleicht Bremen? GB 5.36 y (emojis) 
rieda Frieda: 
Sie hat man schon vorgeschlagt - It has already been suggested — (name 
ip hat ‘nein’ gesagt. 18:4! of the teacher) said ‘no’. 
Jürgen Jürgen: 
Es tut mir leid! 18:42 Iam sorry! 
Frieda Frieda: 
Mir auch! 18:43 Me too! 
Emma . e Emma: 
Ja es muss Berlin sel ich vergesse | ves, it has to be Berlin, I always forget 
immer die neue Lánder unseren the ‘new’ countries to our eastern 
óstlichen Gebrüder 23:0 brothers 
Teacher Teacher: 
Sehr gut alle! hat Bremen Very good everyone! (Emma) 
erwáhnt, und hat richtig mentioned Bremen and (Jürgen) 
geahnt. Bremen ist es! " Wer ist guessed right. Bremen is it! (emoji) 
. p" ] Who is it now? (emoji) 
jetzt dran? (©) T 


Figure 4.1 (Continued) 


As regards learners! engagement, Figure 4.1 shows that they responded to the 
writing task but engaged in no other type of peer-to-peer interaction. In all six writ- 
ing tasks uploaded by the teacher, the learners followed the same participation pat- 
tern, i.e., sharing their answers individually and responding directly to the teacher. 


4.3.4 Example 2: *Guess the city" 


The second communicative event “Guess the city activity" presented a different type 
of activity, which was selected because it received a significant number of responses 
from the learners (i.e., responsiveness). In this activity, each learner was asked to describe 
a city in a German-speaking country for others to guess which city it was (see Figure 
4.2). Seven (out of eight) learners were engaged in this activity who posted a total of 
32 individual chat entries. Out of all WhatsApp activities, the “Guess the city" activity 
was the one generating the higher number of responses from the learners. The teacher 
also participated in this communicative event by sharing another 27 chat entries to 
praise learners’ effort (n=2), to prompt them to describe their city (n=3) and to provide 


Teacher 
A new activity this week.. Choose a 
destination in Germany or Austria 
but don't tell the rest of the group. 
Imagine you're going to travel there. 
Write to the group describing its 
geographical location, how you will 
travel, what is special about this 
place and why you suggested it. The 
rest have to guess where it is. 
Here goes... 
Die Stadt liegt im Norden 
Deutschlands. Ich werde dorthin 
fliegen. Die Stadt ist zugleich Stadt 
und Bundesland. Ich habe diese Stadt 
gewihlt, weil ich als Student da war, 
und immer nicht Freunde dort habe. 
Wo werde ich sein? ©) rie 
) Helmuth 

Es ist Hannover? 11:53 


Frieda 

Ich glaube, dass die beiden 

von Hamburg und Bremen 

Bundeslande sind, und also ist sie 

eine dieser Stádte? 12:10 

Helmuth 

Ich bin beantworten falsch, aber 

Hannover war ein Stadt und 

Bundeslànde für 92 Tages einmal © 
1227 


Frieda 


e .. 


Emma 
Diese Stadt könnte entweder 
Hamburg oder Bremen sein. Ich 
glaube, dass Bremen keinen groBen 
Flughafen hat. Daher wühle ich 


Teacher: 


A new activity this week... Choose a 
destination in Germany or Austria but 
don’t tell the rest of the group. Imagine 
you're going to travel there. Write to the 
group describing its geographical 
location, how you will travel, what is 
special about this place and why you 
suggested it. The rest have to guess 
where it is. Here goes... 


The city is in the north of Germany. I 
will fly there. The city is both a city and a 
federal state. I chose this city because I 
was there as a student and always have 
friends there. Where will I be? (emoji) 


Helmuth: 
Is it Hanover? 


Frieda: 


Ithink that Hamburg and Bremen are 
federal states, so is it one of those cities? 


Helmuth: 


Tm answering wrong, but Hanover was a 
city and a state for 92 days once (emoji) 


Frieda: 
(emoji) 


Emma: 


This city could be either Hamburg or 
Bremen. I don't think Bremen has a big 
airport. That's why I choose Hamburg 
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Keiner hat bisher richtig 
geantwortet... (E) 


Frieda 
Es muss Berlin sein - die dritte Stadt, 
die auch ein Bundesland ist? 17:56 
Jürgen 
Vielleicht Bremen? (Ql... 
Frieda 
Sie hat man schon vorgeschlagt - 

hat 'nein' gesagt. 18:41 


Jürgen 

Es tut mir leid! 18:42 

Frieda 

Mir auch! 18:43 

Emma 

Ja es muss Berlin sein, ich vergesse 


immer die ‘neue’ Länder unseren 
östlichen Gebrüder 


23:06 


Teacher 
Sehr gut alle! hat Bremen 
erwähnt, und hat richtig 


geahnt. Bremen ist es! g Wer ist 
jetzt dran? ©) 


0901 


53 


Teacher: No one has answered 
correctly so far...! (emoji) 


Frieda: 


Tt must be Berlin - the third city that is 
also a federal state? 


Jürgen: 
Maybe Bremen? (emojis) 

Frieda: 

It has already been suggested — (name 
of the teacher) said ‘no’. 

Jürgen: 

Iam sorry! 

Frieda: 

Me too! 


Emma: 

Yes, it has to be Berlin, I always 
forget the ‘new’ countries to our 
eastern brothers 


Teacher: 


Very good everyone! (Emma) 
mentioned Bremen and (Jürgen) 
guessed right. Bremen is it! (emoji) 
Who is it now? (emoji) 


(emoji) 


Hamburg (3) 


1309 


Figure 4.2 "Guess the city activity" (part I). 


feedback to their contributions (n=22). Overall, a total of 59 individual chat entries 
were made in this activity, which lasted for nine days. Due to the length of this com- 
municative event, in Chapter 4 we provide two of its parts. The first part (see Figure 
4.2) concerned how the teacher structured and initiated the activity and showed how 
the learners engaged with the teachers initial post. The second part (see Figure 4.3) 
involved one of the learners description of a city and his peers’ contributions. 

In this communicative event, the teacher posted the instructions of the activity 
using English and then shared a description of his selected city in German. This 
activity caught learners' attention and nine minutes after the teacher's initial post, 
Helmuth replied and made a guess regarding the city described by the teacher. In 
response to the teacher’s post, four learners (Helmuth, Frieda, Emma and Jürgen) 
used the target language and tried to identify the correct city. Learners also tried to 
negotiate and renegotiate to reach an answer by sharing their own interpretation 
of the clues and using their knowledge to argue for or against a suggested answer 
(see Figure 4.2). Once all the learners shared their guesses, the teacher revealed the 
correct answer and used a thumbs-up emoji gesture indicating approval of their 
efforts. The teacher ended this online exchange by prompting the learners to 
describe their city “Wer ist jetzt dran?" (whose turn is it now?). 

Following the teacher's prompt, Otto took the initiative and shared a description 
of his city (see Figure 4.3). Frieda made a guess about the city in question, which 
however was not the right one as indicated by Otto. Once he provided more infor- 
mation about his selected city, Emma followed-up with another guess, which Otto 
confirmed was the correct one. 

Overall, this activity encouraged learners to read their peer's contributions, iden- 
tify the clues provided and then guess the city. It also allowed them to interact with 
each other. 
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4.3.5 Example 3: "Your Bank Holiday weekend” 


The third communicative event concerned another activity posted by the teacher 
and involves learners' self-reflection upon their sharing practices in the online 
environment (1.e., reflexivity) which emerged from the interview data. To begin 
with, three (out of twelve) of the activities uploaded by the teacher required learn- 
ers to provide peer-feedback. Specifically, the teacher asked learners to comment 
on their peers’ sentences and/or correct them. It was observed that even if the 
learners replied individually to the activities, none of them left any comments 
related to the learners' language use in any of the chat entries posted by the learn- 
ers. As Figure 4.4 shows, the teacher instructed the learners to describe what they 
had done over the bank holiday weekend, including a clear prompt for the learners 
to comment on each other’s contributions and/or correct them. The teacher fur- 
ther stated that he would also provide feedback to the learners’ contributions. The 
teacher followed the same pattern in all activities that required peer-feedback, and 


Otto Otto: 


Diese Stadt liegt in der Mitte This city is in the middle of Germany. 
Deutschlands. Es ist bekannt für lt is known tor its baroque buildings 

: M and the cathedral has the tomb of Saint 
seine barocken Gebäude und der Boniface - a big hint! You have to fly 
Dom hat das Grab des Heiligen to Frankfurt first and then take the 
Bonifatius - ein großer Hinweis! Man train. 
muss zuerst nach Frankfurt fliegen 


und dann mit dem Zug fahren. 13:20 
. : And - I visited this city with my 
Und - ich habe diese Stadt German friend many years ago. 


vor vielen Jahren mit meinem 
deutschen Freund besucht. 


3:27 
Frieda: 

Frieda Could it be Cologne? 
Könnte sie Köln sein? 13:35 
Otto Otto: 
Nein- Es ist kleiner als Köln und liegt No - it is smaller than Cologne and is 
nordöstlich von Frankfurt. 13:39 located northeast of Frankfurt. 
Emma 

m TN Emma: 
Der heiligen Bonifatius stammte 
aus Der Grafschaft Devon, ein Saint Boniface came from the county 


of Devon, a local Saint. So, the city 


einheimischer Sankt, NSO Was must be Fulda. I've never been there 


So, muss die Stadt Fulda sein. Ich 


. : before 
war bisher nie dort 13:58 
Otto Otto: 
Ah richtig Peter! 14:01 Ah right (name of the student)! 


Figure 4.3 "Guess the city activity" (part 2). 
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Otto 


Diese Stadt liegt in der Mitte 
Deutschlands. Es ist bekannt für 
seine barocken Gebäude und der 
Dom hat das Grab des Heiligen 
Bonifatius - ein groBer Hinweis! Man 
muss zuerst nach Frankfurt fliegen 
und dann mit dem Zug fahren. 


Otto: 


This city is in the middle of Germany. 
It is known for its baroque buildings 
and the cathedral has the tomb of 
Saint Boniface - a big hint! You have 
to fly to Frankfurt first and then take 
the train. 


13:20 
: s And - I visited this city with my 

Und - ich habe diese Stadt German friend many years ago. 
vor vielen Jahren mit meinem 
deutschen Freund besucht. 13:27 

Frieda: 
Frieda Could it be Cologne? 
Könnte sie Köln sein? 13:35 

Otto: 


Otto 


Nein- Es ist kleiner als Köln und liegt 
nordöstlich von Frankfurt. 13:39 


Emma 

Der heiligen Bonifatius stammte 
aus Der Grafschaft Devon, ein 
einheimischer Sankt, na so was. 
So, muss die Stadt Fulda sein. Ich 
war bisher nie dort 13:58 


Otto 


No - it is smaller than Cologne and is 
located northeast of Frankfurt. 


Emma: 


Saint Boniface came from the county 
of Devon, a local Saint. So, the city 
must be Fulda. I've never been there 
before 


Otto: 


55 


14:01 Ah right (name of the student)! 


Ah richtig ! 


Figure 4.3 (Continued) 


Figure 4.4 provides a representative example of how the teacher structured and 
initiated these activities. 

Two days after the initial post Jürgen replied to the activity by sharing a picture 
of him and Helmuth who were having holidays in Berlin. Louisa and Otto shared 
a chat entry in response to Jürgen’s picture, and three days later, Louisa replied to 
the activity by describing what she had done over the bank holiday weekend. The 
teacher then followed up with a comment related to the uploaded picture, thank- 
ing Jürgen and Helmuth for sharing, and prompting the learners to discuss their 
trip in Germany during their in-classroom lesson. Emma was the last learner to 
reply to the activity by describing what she had done during the bank holiday. 

As shown in Figure 4.4, the learners partially followed the task instructions and 
their contributions were limited to providing individual answers to the activity. 
The teacher gave time to the learners to comment on their peers’ contribution and 
when this did not happen, eight days after the initial post he provided feedback to 
the ones who participated in this. Similar to this activity, all the other activities of 
this type did not involve any peer-feedback, indicating that the learners in this 
study demonstrated a reluctance to provide feedback to their peers. 
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This point was raised in the interviews, where four questions were asked seeking 
to examine how learners self-reflect upon their sharing practices. Accordingly, 
interviews were used to identify the reason(s) why learners in this study 
demonstrated a reluctance to provide peer-feedback. Interview data suggested that 
learners did not feel comfortable about providing feedback, because it was perceived 
as potentially offensive, as evidenced by Louisa in the following: 


I wouldn't want to offend them by saying oh I think you should have done 
this or you could have done that I feel uneasy about that to be honest. 
(Louisa) 


Indeed, learners did not" “like to point the finger at anybody (...) because you [they] 
value that they have taken time to do something” (Otto). What is more, learners (n=3) 
felt that they were not entitled to provide peer-feedback as they were not profi- 
cient enough in the target language and that discouraged them from doing so. 
These views are articulated in the following quote: 


OLIN) Teacher: 
Teacher Another group activity this week... In three 


ad 2 is UE He Ms A ar short sentences say what you did over the 


did over the Bank Holiday weekend in 
German. I'd like others in the group to 
comment on your sentences and / or 
correct them. Then I'll give feedback 

too. It'll be nice to have a contribution 


from each person if possible. © 


10 MAY 2019 


Jiirgen 


Hallo von Berlin g = 13:28 


Louisa 
xm n [IDE 13:31 
Otto 


Ah das Brandenburger Tor! (55) 
Gibt es Souvenirverkáufer in der 
Nahe? &3 


Bank Holiday weekend in German. I'd like 
others in the group to comment on your 
sentences and / or correct them. Then I'll 
give feedback too. It'll be nice to have a 
contribution from each person if possible. 
(emoji) 


Jürgen: 


(Image) 


Hello from berlin (2 emojis) 


Louisa: (7 emojis) 


Otto: Ah the Brandenburg Gate!(emoji) 
Are there souvenir sellers nearby? (emoji) 


Figure 4.4 Activity involving peer-feedback. 
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13 MAY 2019 


Louisa Louisa: 
Letzte Wochenende... 5) Am 
Samstag Morgen habe ich Fußball 
gesehen ©. Die Junge haben 


gewonnen und mit diesen drei 
Punkten die MK-Gebiet Liga gesiegt 


6. Natürlich war das eine Feier für 


Last weekend ... (emoji) On Saturday 
morning I watched football (emoji). The 
boys won and with these three points they 
won the MKarea League (emoji). Of 
course, this was a celebration for us (emoji) 
and yes, we (parents, trainers, fans) drunk a 
lunchtime beer (emoji). In the evening I 
cooked paella (emoji) and watched a little 
TV afterwards (emoji). On Sunday I 

Abend habe ich Paella gekocht aj worked in the garden (2 emojis), because 
und danach ein bisschen Fern theweather was so nice (2 emojis) 

gesehen B. Am Sonntag habe ich 


uns S und ja wir haben (die Eltern, 
die Trainers, andere Fans) ein 
Mittagszeitbier getrunken @. Am 


im Garten gearbeitet ® e$, weil das 


Wetter so schön war 4> gdb. 


09:30 


Louisa 
Louisa: 
(Image) 
(e 
x g Y 09:31 (3 Emojis) 
14 MAY 2019 
Teacher 
Vielen Dank für das Foto und Teacher: 


! Das Brandenburger Tor ist 
wirklich sehenswert, nicht wahr?! Ihr 
könnt uns heute abend vielleixht 
mehr über Eueren Kurztrip erzählen?! 


© 06:58 


Emma 


Thanks for the photo (Jürgen) and 
(Helmuth)! The Brandenburg Gate is really 
worth seeing, isn't it?! Maybe you can tell 
us more about your short trip this evening?! 


Am Morgen des Maifeiertags spielte Emma: 

ich Federball e mit meinen On the morning of May Day, I played 
Bekannten, die auch Mitglieder der badminton (emoji) with my friends who are 
Towcesterer Universitat des dritten also members of Towcester University of 
Zeitalter sind. Am Abend des the Third Age. On the evening of May Day, 
Maifeiertags ging ich mit meiner Frau my wife and I went to Little Brickhill to do 
nach Little Brickhill, um Square some square dancing (emoji)because it's 
Tanzen ** zu treiben, weil es uns lots of fun for us. 


NN 


viel Spaß macht. 


20:23 


Figure 4.4 (Continued) 
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I wouldn't contribute to that because I don't feel that my knowledge is enough 
to give people feedback. 


(Jürgen) 


Another reason revealed by the interview data was related to the teacher’s role. 
Specifically, two learners seemed to think that “that’s more [name of the teacher]’s role 
to do that it’s a bit uncomfortable sometimes” (Otto), also evidenced in the following: 


I was quite happy to suggest things and correct them when we were working 
through some exercises if they welcomed it or not I don’t know but I was 
very anxious not to undercut [name of the teacher]’s role who is obviously 
the teacher. 

(Emma) 


All the above views that emerged from the semi-structured interviews shed light 
on learners’ reluctance to provide feedback to their peers. In what follows, we analysed 


the communicative events in terms of “selecting”, "styling", and "negotiating" as 
discussed in Section 4.3.3. 


4.3.6 Sharing practices: selecting, styling, negotiating 


The discussion so far has provided three examples of the activities uploaded by the 
teacher into the WhatsApp group and showed how learners participated in and 
contributed to these. In terms of selecting, the language learning activities dis- 
cussed in this study revealed a recurrent pattern in the sense that they all aimed to 
enhance learners’ writing competence and vocabulary acquisition. 

As regards styling, it was evident that the activities discussed above involved dif- 
ferent entextualisation patterns. Specifically, the first activity (see Section 4.3.3) was 
introduced using the German language, while the instructions in the other two 
(see Sections 4.3.4. and 4.3.5) were given in English. The teacher followed the 
same styling format in all activities (n=6) which required learners to compose 
sentences using given vocabulary. On the contrary, when sharing activities with a 
certain degree of complexity (e.g. “Guess the city activity") he styled the instruc- 
tons using the English language. Moreover, when introducing activities that 
required learners to provide peer-feedback (see Section 4.3.5), the German teacher 
used English. The teacher's choice of language in these activities was possibly made 
to ensure that the learners had understood the language task and what was expected 
from them. 

The next, and most important, level of analysis was "negotiating", which con- 
cerned learners’ engagement with the shared activities. The communicative events 
discussed above showed that learners’ responses to the uploaded activities gener- 
ated different degrees of engagement. In the simplest case, learners’ participation 
was limited to responding directly to the teacher (see Section 4.3.3), while on 
other occasions (see Section 4.3.4) learners were engaged in more substantial ways. 
More specifically, and in terms of “selecting”, those activities which prompted 
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learners to reply individually to the teachers post, and focused on the abstract 
practice of language form, resulted in no interactions amongst the members of the 
group. On the contrary, the “Guess the city" activity encouraged learners not only 
to share their individual responses but also to read their peers’ contributions in 
order to decipher the clues and then provide their answers. This activity also 
resembled a game (or a quiz) which is rooted in learners’ meaningful experience 
of Germany and necessitates learners to draw on their personal knowledge, share 
their own interpretation of the clues, negotiate these clues with their peers, and 
finally reach to the correct answer. 

As regards peer-to-peer feedback, and in terms of “negotiating”, the discussion 
above indicated that learners demonstrated a reluctance to engage with such 
interactions. Apart from the reasons which emerged from the analysis of the 
interview data, analytic attention should be placed on the "styling" of the 
instruction in these types of activities. Specifically, as shown in Section 4.3.5, the 
teacher explicitly asked the learners to comment on their peers’ sentences and/ 
or correct them. The exact same instructions were provided in all activities that 
required learners to provide feedback to their peers. Moreover, in terms of 
styling, each of the three activities was divided into two parts. The first part 
involved the language task prompting learners to reply individually to the activity 
(e.g. what you did over the Bank Holiday weekend?) and the second part directly 
asked the learners to comment on or correct their peers’ contributions. The 
design of the tasks prompted learners to provide feedback in an overt way, while 
the first part of the tasks did not require learners’ interaction. Consequently, the 
selection and the styling of the tasks might have not encouraged learners to 
engage in such interactions. 

Finally, regarding "negotiating", the analysis of the communicative events 
examined above showed that participants’ engagement with the shared activities 
were not time-bound. Specifically, some interactions in the online environment 
were synchronous (or near synchronous). For instance, the online exchange dis- 
cussed in Section 4.3.4 lasted for 41 minutes resulting in a total of six chat entries. 
This showed that WhatsApp can enable synchronous (or near synchronous) com- 
munication and emphasises the potential for immediate interactions in this plat- 
form. Nonetheless, in the same activity (Section 4.3.4), participants were sharing 
their contributions for nine days. Such observations suggest that the MIM appli- 
cation can also afford asynchronous and/or diachronic communication. Moreover, 
since WhatsApp is an archived online environment, it can enable learners to 
access previous chat entries retrospectively and re-read as well as decipher what 
other users had shared. This appears to be a significant benefit of using WhatsApp, 
since "You can search back (...) you can look back into the Intermediate German 
WhatsApp group on your phone or iPad or whatever and you can find the conversation so 
that was quite good" (Otto). In a traditional in-classroom learning environment, 
where learners inevitably do not have the chance to archive any interactions, they 
are not able to access the learning context over time. In this study, the use of 
WhatsApp allowed learners to access the discussion threads as well as the learning 
material at any point. 
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4.4 Discussion and moving forward 


The purpose of Chapter 4 was to study patterns of interaction in WhatsApp, 
exploring the extent to which this MIM application might provide a favourable 
environment for language learning. Coinciding with Keogh's (2017) study, the 
findings revealed learners’ reactive and limited participation, while the activity in 
the WhatsApp group heavily relied on the teacher efforts to initiate conversation. 
A reason for these participation patterns emerged from the analysis of the inter- 
view data. Specifically, learners perceived that only the teacher had the authority 
to initiate conversation in WhatsApp. Such perceptions and predefined expecta- 
tions highlight the different dynamics of participating and sharing when using 
WhatsApp for formal (or non-formal) language learning as compared to informal 
everyday life use. 

As regards the affordances of the platform, this study showed that participants 
were able to share their contributions without any temporal constraints. Specifically, 
it was evident from the analysis of the selected communicative events that 
WhatsApp can afford synchronous (near synchronous), asynchronous and/or dia- 
chronic communication. Another important benefit of WhatsApp indicates that 
the archived nature of the platform enabled participants to retrieve previous chat 
entries retrospectively and re-read as well as decipher what other users had shared 
at any point. 

The analysis of the communicative events examined in this study suggests that 
what is being shared (selecting), and how this 1s done (styling) can influence the 
ways in which participants engage with the shared content (negotiating). As regards 
online interaction, other studies (see Aburezeq & Ishtaiwa, 2013; Andujar, 2016; 
Keogh, 2017) emphasised that WhatsApp can encourage language interaction 
among participants. Notwithstanding that this MIM application can afford lan- 
guage interaction, this study showed that learners’ participation and interaction 
cannot be ensured by the mere use of the platform, but heavily relies on task 
design. Moreover, the effective selection and the styling of the language activities 
can potentially stimulate other interactions, such as peer-to-peer feedback. 

Therefore, we argue that the educational value of this MIM application and its 
appropriateness as a teaching and learning environment need to be further investi- 
gated. Future research should aim to establish a clear theoretical framework for the 
integration of WhatsApp into the language classroom by developing transparent 
guidelines and pedagogical strategies which can lead to an effective use of the 
medium for language teaching and learning purposes. 


4.4.1 Implications for practice 


Chapter 4 explored the use of WhatsApp by a group of adult learners of German 
language and their teacher in an adult education centre. The findings suggest that 
the mere use of WhatsApp cannot guarantee language interaction among partici- 
pants. Moreover, inflated expectations that WhatsApp can generate more interac- 
tion in language settings because of its communicative use in everyday life are 
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questioned. Consequently, current pedagogical practices cannot be reformed sim- 
ply by exploiting the familiarity and popularity of this MIM application. WhatsApp 
can afford online language interactions and might effectively facilitate language 
learning, but this necessitates extensive training of educators to use the medium to 
its full potential. Therefore, practitioners should consider the process of planning, 
designing, structuring, and implementing educational activities in WhatsApp and 
should not simply assume that the use of WhatsApp will magically increase learn- 
ers’ participation and interaction. 
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Chapter 5 


First steps towards self-regulated 
learning 


Setting goals in MOOCs 


Barbara Conde Gafaro 


5.1 Introduction 


I am up for something new. I don't want to be a dinosaur and I want to keep up with 
things, but I just needed a little bit of help to get started. 
Irene, adult language learner 


The COVID-19 pandemic has disrupted our way of learning and teaching. 
Lockdown measures implemented by governments to prevent the spread of the 
virus have changed our everyday life, including education. These measures have led 
most educators and learners to step into the world of online learning and opt for 
learning technologies to replace face-to-face learning environments. Although the 
work presented in Chapter 5 happened before COVID-19, this contribution’s 
narrative is even more relevant at this time of the pandemic. Learners are expected 
to be equipped with strategies and study skills to chart a path for their sudden 
online education without having the constant supervision of an in-class teacher, 
which is similar to the cases discussed in this chapter. 

These unprecedented circumstances have resulted in various opportunities as 
well as challenges. Learners have the option to assume a responsible role in their 
online education. When they take responsibility for their learning, they regulate 
their thoughts, feelings, and actions, i.e., learners become masters of their learning 
processes for attaining goals (Zimmerman, 2011). Employing self-regulatory pro- 
cesses and being aware of how these processes influence one’s preparation and 
willingness to self-regulate are crucial steps in today’s open world learning. For 
example, learners are anticipated to self-regulate their learning in Massive Open 
Online Courses (MOOCs) (Milligan & Littlejohn, 2016), since “a MOOC is 
completely voluntary. You decide that you want to participate, you choose how to 
participate, then you participate” (Downes, 2012). However, taking part in such 
online courses becomes a challenge for those who cannot learn independently 
(Littlejohn & Hood, 2018).Therefore, learners in order to fully benefit from their 
studies should be independent and know how to self-regulate to succeed in 
learning at a distance. 

In foreign language education, MOOCs have been considered to support lan- 
guage learning at a distance (Gimeno-Sanz, 2021). These online courses provide 
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opportunities for language learners to practise their target language, either by fol- 
lowing courses that are designed to teach a language (LMOOCSs) or by selecting 
MOOCs that are related to learners’ interests so that they can study a language for 
specific purposes, see also Chapter 6 (Chua, 2022). Likewise, the pandemic has 
influenced a growing global demand for remote language learning. A recent review 
of MOOC stats and trends listed foreign language learning in the top ten of most 
popular subjects studied amidst the pandemic — with 815 language courses offered 
by the main providers at the time of writing (Class Central, 2021; Shah, 2020). Yet, 
the challenges that MOOCs represent for language learners remain under- 
researched (Gillespie, 2020; Sallam, Martín-Monje, & Li, 2020). The challenge 
mentioned above about self-regulation in MOOCs is not much studied in relation 
to languages (Alonso-Mencía et al., 2020), especially the forethought processes that 
language learners are anticipated to adopt so that they can initiate their self- 
regulated learning in those online courses. 

Hence, Chapter 5 aims to provide a deeper understanding of the forethought 
processes, with particular reference to goal setting and goal orientation. Both pro- 
cesses were employed by 19 adult language learners during four weeks of engage- 
ment with MOOCs as part of their classroom-based language courses. The findings 
from Chapter 5 may contribute to support learners’ self-regulated learning and, last 
but not least, encourage learners to assume responsibility for their language educa- 
tion in which active learning is the new normal. 


5.2 Self-regulated learning 


Self-regulated learning (SRL) is conceptualised as a dynamic group of processes 
that learners employ to initiate, sustain, and assess their learning towards goal 
achievement. Zimmerman (1989) was one of the first to cover the aspect of 
metacognition in his triadic model of self-regulation. Metacognition refers to 
two clusters of activities, namely learners’ self-awareness of how, when, and where 
to use different cognitive strategies and the regulation of those strategies that 
direct their learning (Flavell, 1979). Winne and Hadwin's (1998) SRL model also 
included this aspect of learning; however, it failed to consider the interaction of 
metacognitive processes with other motivational and social aspects of self- 
regulation (Greene & Azevedo, 2007). This missing interplay evidenced in the 
latter was presented in a redefined SRL model posed by Zimmerman and Moylan 
(2009). They designed an integrative three-phase model to explain potential 
interactions among metacognitive and motivational processes that occur during 
learning efforts. 

Zimmerman and Moylan’s (2009) cyclical model of SRL introduces the notions 
of metacognition and motivation throughout a preparatory, a performance, and a 
reflective phase, described below: 


1 Forethought Phase: it refers to learning processes and sources of motiva- 
tion that are contemplated in preparation for efforts to learn and affect 
learners' willingness to self-regulate their learning. 
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2 Performance Phase: it involves processes that are employed during efforts 
to learn and influence learners’ self-control and self-monitoring of their 
performance. 

3 Self-reflection Phase: it refers to processes that follow efforts to learn and 
subsequently influence learners’ reflection and affective reactions to that 
learning experience." These self-reflections, in turn, influence forethought 
regarding subsequent learning efforts, which completes the self-regulatory 
cycle". 


(Zimmerman & Moylan, 2009, p. 301) 


The three self-regulatory phases are composed of 20 metacognitive processes and 
sources of motivation. However, this chapter’s scope focuses only on the processes 
of goal setting and goal orientation included in the forethought phase. The fore- 
thought phase consists of two main categories: task analysis processes and self- 
motivation sources. In the first category, learners are anticipated to unpack a 
learning task, set educational goals, and outline a strategy to be prepared for the 
task and its environmental setting. Goal orientation is part of the second category 
and reflects learners' beliefs about the purposes of engaging in learning or per- 
forming tasks (Zimmerman & Moylan, 2009). 

The SRL cyclical model places goal setting at the top of the learning processes that 
learners are anticipated to deal with in the forethought phase. Setting goals enables 
learners to initiate their learning and monitor their progress towards those goals and 
adjust their learning, if necessary (Zimmerman, 2000). There are two types of goals 
identified in the socio-cognitivist literature: distal (long-term) goals and proximal 
(short-term) goals. Several studies show that setting proximal goals is more effective 
than focusing on distal goals (Zimmerman, 2008). The most effective proximal goals 
are challenging for learners, specific to the task, and align with other goals (Zimmerman, 
2008). Altogether, research suggests that learners should set specific outcomes in time, 
either driven by learning or performance-oriented goals, to learn more effectively on 
their own. The following section expands on the importance of goal setting in flexible 
learning environments by covering works that examine successful learning in MOOCs. 


5.3 MOOCs and goal setting 


MOOCs appeared in online education when Siemens, Downes, and Cormier facili- 
tated a way of learning in the networked world to a total of 2,200 people via an online 
course called “Connectivism and Connective Knowledge (CCK08)” (Downes, 2009). 
MOOCs have continued to provide large-scale participation and access to subject- 
specific resources via the web since then. In the case of LMOOCs, these have been 
designed for anybody interested in learning particular aspects of a foreign language due 
to the limited time they are offered, between four and six weeks (Gimeno-Sanz, 2021). 

Although MOOCs represent the growth of online learning delivered at a mas- 
sive scale, their principle of open access to learning for everyone has been ques- 
tioned (Littlejohn & Hood, 2018), and subsequently, their commitment to open 
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world learning. MOOCs welcome people worldwide to access online education 
content without any academic prerequisite needed. However, the access to discus- 
sion forums (Chua, 2022), multimedia material and quizzes offered in those 
courses is limited to people who can learn independently; see also Chapter 8 
(Rizvi et al., 2022) and Chapter 9 (Iniesto et al., 2022). 

MOOC learners have the advantage of choosing their learning path (instructor- 
paced or self-paced online courses), managing the resources they want to cover, 
and the time they want to invest in the course materials (Beaven, 2013). However, 
such flexible learning approach embedded in the design of MOOCs might only 
favour “those who are able to self-regulate their learning, leaving the most disad- 
vantaged behind” (Littlejohn & Hood, 2018, p. 31). By way of illustration, learners 
who have formal academic qualifications typically enrol in and complete MOOCs 
at relatively higher rates (Kizilcec, Saltarelli, Reich, & Cohen, 2017). Similarly, uni- 
versity students with a master’s degree or PhD often report higher levels of goal 
setting, strategic planning and task strategies than people with lower qualifications 
enrolled in MOOCs (Kizilcec, Pérez-Sanagustin, & Maldonado, 2017). Therefore, 
SRL becomes a crucial tool kit for learners who want to access online educational 
content and achieve success within such learning environments. 

Likewise, Gimeno-Sanz (2021, p. 53) also argues that in most cases, MOOC 
learners “have to self-regulate their learning, very much relying on cognitive and 
resource management strategies”. Goal setting is one of the self-regulatory pro- 
cesses employed by successful learners in those online courses.A recent study that 
surveyed 643 MOOC learners found that “MOOC completers reported signifi- 
cantly higher use of the goal-setting SRL subprocess than did MOOC non- 
completers” (Handoko et al., 2019, p. 50). The findings are aligned with previous 
studies that identified goal setting as a common metacognitive process among 
successful learners in MOOCs (Kizilcec, Pérez-Sanagustin, & Maldonado, 2017; 
Milligan & Littlejohn, 2016). These studies examined the types of goals (proxi- 
mal/distal) and how they influenced learning in MOOCs related to educational 
technology and STEM subjects. However, little is known about how learners set 
goals in LMOOCS or specific content-based MOOCs, particularly when prepar- 
ing to work on these online courses as part of their classroom-based language 
courses. This gap in the literature raises the question of how language learners set 
their goals when engaging with MOOCs to support their classroom-based lan- 
guage education, and that was the main aim of the empirical study reported in 
this chapter. 


5.4 Research methods 


The empirical study described in this chapter was framed within a multiple-case 
study research. A case study examines a contemporary phenomenon in depth and 
within its real-world context (Yin, 2018). The phenomenon, also commonly 
known as the case, can involve persons, events, or decisions (Thomas, 2011). A 
multiple-case study comprises two or more cases to gain a detailed understanding 
of a situation. Examining multiple cases also contributes to having richer and more 
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rigorous findings than the ones obtained solely based on a single case. Hence, con- 
ducting a multiple-case study was suited to capture the complexity of learners’ 
goal-setting processes in MOOCs based on different language learners’ experi- 
ences. As explained by Thomas (2011, p. 513), case studies can be "studied holisti- 
cally by one or more methods”. The design of the multiple-case study reported in 
this chapter employed multiple research methods that included three quantitative 
and qualitative instruments: 


1 Four weekly monitoring surveys (WSURV1/2/3/4), with mostly open- 
ended questions, administered during each week of engagement with the 
MOOCSs. The second question in the WSURV (what was your learning goal for 
this week?) allowed the researcher to delve into the type of goals participants 
set for the online courses. 

2 An online SRL questionnaire (SRLQ) sent in week five of the project to 
survey participants’ SRL processes. The SRLQ consisted of 29 items in total, 
but this chapter focused only on the first three that dealt with goal setting 
processes (1. I set specific short-term (daily or weekly) learning goals for the MOOC 
I chose. 2. I set specific long-term learning goals (monthly or for the whole MOOC). 
3. I set realistic deadlines for learning in the MOOC.). The study did not aim to 
compare participants’ SRL processes before and after their engagement with 
the MOOCs. Thus, the questionnaire was only administered at the end of the 
learning process. 

3 A semi-structured interview (INV) conducted at the end of the project to 
probe participants’ forethought processes of goal setting and goal orientation 
in their online courses. 


The SRLQ was adapted based on the Motivated Strategies for Learning 
Questionnaire (Pintrich et al., 1991) and the INV was an alteration of an interview 
designed by Littlejohn and Milligan (2015).The research design of a multiple-case 
study follows a “replication logic”, i.e., cases replicate the exact conditions of the 
first case to predict similar or contrasting results based on anticipated reasons (Yin, 
2018). The extent of the replication logic presented in this chapter involved the 
exact three instruments to collect data in each case. The cases also shared the fol- 
lowing conditions: small groups of adult language learners attending face-to-face 
language courses while engaging with MOOCs for a month to enhance that 
classroom-based instruction. The learners within those groups were the cases in 
this multiple-case study research. 

Specific characteristics were considered within the selection criteria of the cases. 
The researcher contacted gatekeepers who run language courses for adult learners 
and included independent learning and/or interactive technologies as part of their 
syllabi. The 19 participants, who voluntarily joined the multiple-case study research, 
were learning languages for different purposes in two different contexts. In Case 
study 1, ten participants were taking face-to-face language courses in Spanish, 
Italian and French at a community learning centre in Milton Keynes (UK). The 
gatekeeper in Case study 1 regularly asked learners to complete a learning plan and 
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record of achievement where they had to write their learning goals and marked if 
those were achieved by the end of the course. In Case study 2, nine participants 
were taking an English for academic and professional purposes course (ESP) 
offered as part of a local association in Ferrara (Italy). The gatekeeper contacted in 
Case study 2 usually asked students to use MOOCs within her ESP lessons to 
work on their motivation and academic performance. The gatekeepers' familiarity 
with the study's core ideas, goal setting processes and MOOCs, diminished acces- 
sibility issues in the recruitment process. 

All participants were asked to engage with a MOOC of their choice, since stu- 
dents who find personal interests in a learning task are more likely to regulate their 
learning (Zimmerman, 2000). They were asked to use Class Central, a search 
engine tool to browse MOOCs by subjects and providers. Whereas learners in 
Case study 1 selected LMOOCs that suited their intermediate language profi- 
ciency level and personal interests, learners in Case study 2 chose specific content- 
based MOOCS that matched their disciplinary specialisms and advanced language 
proficiency level. At the start of the project, the former self-reported a lower lan- 
guage proficiency level than the latter. This advantage in terms of language exper- 
tise enabled the ESP learners to select their MOOCs from a wide range of online 
courses that moved beyond the linguistic content usually offered in LMOOCs. 
Once participants selected their courses, they were free to decide how and when 
to work with the online material, though a minimum of two hours of study per 
week was suggested. They were also given pseudonyms, whose initial letter indi- 
cated which of the languages they were studying. 

A substantial critique of case study research is that cases are not sampling units; 
hence, they cannot be generalisable. This perceived limitation can be addressed by 
connecting the case study to a theory so that “analytic generalisations” can be 
made, i.e., expanding theories at a higher conceptual level rather than extrapolating 
probabilities with “statistical generalisations” (Yin, 2018). Therefore, the researcher 
identified and classified learners’ goal setting and goal orientation processes follow- 
ing a deductive approach within the qualitative data analysis based on Zimmerman 
and Moylan’s (2009) forethought processes. The qualitative information was 
triangulated with the responses from the SRLQ to answer the question raised in 
this chapter. Altogether, the study’s research design used three instruments to 
establish a chain of evidence concerning the goal setting and goal orientation 
processes that 19 participants in two case studies employed while engaging with 
MOOCs during four weeks of their face-to-face language courses. 


5.5 Results 


5.5.1 Goal setting of community-based language learners 
(Case study I) 


Participants were asked to reflect on their goals in the second question of the weekly 
monitoring surveys (WSURV), in which they had to write down the learning goal 
for each week of their LMOOC-based learning. All ten participants set goals that 
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covered the revision of grammar topics, vocabulary learning and the mastery of lan- 
guage skills (mainly listening skills). Another common pattern found around goal 
setting was learners’ preferences for specific proximal goals. Figure 5.1 indicates a 
preponderance of specific-short term goals’ over specific long-term goals among 
participants. 

Based on the responses from the SRLQ illustrated below, six out of ten partici- 
pants reportedly set specific short-term goals rather than specific long-term goals 
when working with their LMOOCs. Figure 5.1 also shows that most learners 
claimed to have set specific short-term goals alongside realistic deadlines for learn- 
ing in their LMOOCs. The element of time is crucial when setting specific and 
achievable learning goals. Unfortunately, there was not much evidence in the qual- 
itative data that showed learners formulating specific and time-limited goals during 
their LMOOC-based learning. 

Despite the lack of realistic goals observed in the qualitative data of Case study 1, 
learners often elaborated multiple goals as part of their LMOOC experience. Eight 
out often participants set numerous targets during the four weeks of online learning. 
Most of them were targeting two different aspects of language, and others even 
reported having three goals in one of the weeks. By way of illustration, a learner of 
Italian said in the third weekly survey that she wanted to "revise the perfect tense of 
reflexives [sic] [verbs] and broaden my vocabulary plus continue to practise my 
listening" (WSURV3-Irene). The variety of goals set up by community-based 
learners reflected the different aspects of language they dealt with while engaging 
with the activities and audio-visual content offered in the LMOOCs. 

Regarding participants’ goal orientations, they reported having goals oriented 
towards learning linguistic and sociocultural topics and language skills develop- 
ment. Almost all their goals were mastery-oriented goals aimed at revising an 
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Figure 5.1 Type of goals reported by community-based learners in their LMOOCs 
(Likert Response l-5). 
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Table 5.1 Comparison between mastery-oriented goals and performance- 
oriented goals set by community-based learners 


Mastery-oriented goals Performance-oriented goals 

* "Practise with tenses" * "to get as many chapters finished" 
(WSURV4-Salvador) (WSURV | -Silvia) 

* "to revise reflexive verbs" * "To complete the first week with 
(WSURV2/3-Irene) FutureLearn out and about" 

* "do more listening" (WSURV 1/2/3/4-Sofia) 
(WSURV2/3-Simona). * "To complete week | of the structured 

* "| need to improve my course" (WSURVI/2/3/4-Sarah). 
pronunciation" * "Complete module one about plans" 
(WSURV3-Sarah) (WSURV I -Santos) 

* "Improve my grammar" * "spend more time [on task]" 
(WSURV3-Felix) (WSURV3-Isabella) 


element of grammar, language skills improvement or learning about everyday spo- 
ken vocabulary/specific sociocultural content. Nevertheless, six out of ten partici- 
pants also self-reported performance-oriented goals focused on LMOOC 
completion or performing well in the online course tasks, as illustrated in Table 5.1. 

Learners who often reported specific performance-oriented goals appeared to 
set more realistic deadlines than learners who included mastery-oriented goals as 
part of their online learning experience. For example, Sofia and Sarah set clear 
goals that focused on completing the four weeks of the MOOC (Table 5.1); both 
learners also scored high in the third SRLQ item that measured realistic goal set- 
ting (Figure 5.1). Conversely, Salvador and Felix, who tended to set mastery- 
oriented goals (Table 5.1), presented a low score in the last item ofthat questionnaire 
(Figure 5.1). One of the learners’ mastery-oriented goals reported in week four of 
their online learning, “vocab-pronunciation. Verbs in the past tenses” (WSURV4- 
Salvador), implied some work on the aspects of vocabulary, pronunciation, and 
grammar. However, such work remained unclear and possibly intangible, since it 
did not target a specific outcome to be achieved at a particular point in time. The 
wording of most proximal mastery-oriented goals revealed a vague idea of what 
learners wanted to attain within a specific time limit, questioning the realistic 
nature of these goals that focused on learning rather than performing well in an 
LMOOC. 


5.5.2 Goal setting of ESP learners (Case study 2) 


The nine ESP learners self-reported more proximal goals than distal ones when 
they engaged with specific content-based MOOCs. Most of their self-reported 
goals identified in their four weekly surveys had a clear focus, such as “writing a 
report (one page)” (WSURV4-Elton) or “understanding the deception [MOOC 
topic]” (WSURV3-Erik). A few learners self-reported distal goals that usually 
included future work beyond their MOOCs. For example, Ethan concluded by the 
end of week one:"I think that I should speak a lot: I have already a B2 level and it 
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means I could improve speaking with native speakers. I think it would be the best 
thing I could do" (WSURV 1-Ethan). Likewise, Edwin aimed to use formal expres- 
sions “in a more proper and more conscious way in the future" (WSURV3-Edwin). 
The main pattern of setting clear proximal targets for learning in the MOOCs was 
also evident in the SRLQ responses, as indicated in Figure 5.2. 

Half of the learners in Case study 2 favoured proximal goals within their online 
learning experience, based on the responses from the SRLQ illustrated below. 
Meanwhile, those learners who reported specific distal goals typically formulated 
targets that extended beyond the work with their MOOCs, as previously stated by 
Edwin and Ethan. Most learners also claimed to have realistic deadlines for their 
engagement with specific content-based MOOCs (Figure 5.2). Nevertheless, they 
did not specify time-limited goals when completing the WSURV or taking part in 
the INV. 

Concerning goal orientations, ESP learners reported setting mastery-oriented 
goals that focused on language skills improvement. All nine participants set goals 
aimed at developing receptive and productive language skills. Six out of nine par- 
ticipants also deployed goals that involved learning about the content of their 
MOOCs. For example, participant Elsa explained that her “main aim was to 
understand the basics of that course...how data works and how I can use them in 
my job" (INV-Elsa). Another participant also stressed that ^I needed to improve my 
English competencies, skills, but I also needed to understand what the MOOC 
talked about" (INV-Erik). The mastery-oriented goals for almost half of the par- 
ticipants were twofold: strengthening their target language for academic/profes- 
sional purposes and understanding the MOOC content, which they chose based 
on their specific area of knowledge. 
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Figure 5.2 Type of goals reported by ESP learners in their MOOCs. 
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A few participants also set performance-oriented goals while following their 
MOOCSs. Four out of nine learners focused on completing the online courses or 
outperforming their prior language performance. Elliot, for example, wanted “to 
test my English with a topic I am familiar with" (WSURV 1-Elliot). Similarly, Elton 
aimed at “writing more and better letters, articles and other English texts” 
(WSURV2-Elton) after working on his writing skills in week one of the MOOC. 
Although some learners' goals oscillated between performance and mastery- 
oriented targets, ESP learners commented most frequently on pursuing mastery- 
oriented goals than performance-oriented ones in the weekly surveys and 
semi-structured interview responses. 


5.6 Discussion and moving forward 


Chapter 5 addressed the goal setting and goal orientation processes employed by 
19 adult language learners in two case studies to chart their learning path in 
selected MOOCs. Findings have shed light on how adult learners set goals over 
four weeks of engagement with MOOCs to support their classroom-based lan- 
guage learning, including the type of goals, goal-oriented preferences, and com- 
mon difficulties when setting clear targets. Most learners in both case studies 
reported setting more proximal goals than distal goals. They also recorded more 
mastery-oriented goals than performance-oriented ones in their chosen online 
courses. However, it was difficult for participants to specify time-limited goals 
when initiating their self-regulated learning in MOOCs. 

Community-based language learners and ESP learners formulated more short- 
term goals in their LMOOCs and specific content-based MOOC,, respectively. 
This preference for proximal goals was arguably linked to the delivery mode of the 
online courses, which was described in the platforms that offered the MOOCs. 
There were not many instructor-paced courses available when conducting this 
study, so most participants selected self-paced online courses, i.e., no start or end 
date and less supervision by educators. “This delivery mode affects the way enrol- 
ees work in the course, fostering the establishment of short-term goals (which are 
not necessarily self-defined) that allow learners to persist in the MOOC” (Alonso- 
Mencia et al., 2020, p. 327). The number of weeks in a MOOC and the way con- 
tent is delivered (whether the material is released gradually or from the beginning) 
affect how learners deploy different strategies to self-regulate their learning process 
(Ferguson et al., 2015). Hence, setting proximal goals was a self-regulatory process 
employed by participants to cope with the self-paced learning in selected short- 
term MOOCs. 

All participants also preferred pursuing mastery-oriented goals over performance- 
oriented ones in their chosen online courses. Most learners were not planning on 
completing their MOOCs or obtaining certificates as an outcome of this online 
learning experience. They were formulating goals oriented towards revising spe- 
cific aspects of the language, learning about subjects that were meaningful to them, 
and practising their language skills while covering the audio-visual material. In 
common with others (Beaven, 2013; Gimeno-Sanz, 2021), all participants found 
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opportunities to revise their linguistic knowledge and practise their language skills 
by studying different topics in the target language. 

Nevertheless, participants’ learning goals were not very explicit concerning the 
outcomes they wanted to achieve at a particular point in time. Although they 
claimed to have set realistic deadlines, they did not include a specific time when 
formulating their goals. Not all learners may find the need to set a specific goal in 
a MOOC, particularly if they opt for relying on “predetermined objectives, rather 
than learner-defined goals” (Littlejohn & Hood, 2018, p. viii). However, “goal set- 
ting produces an explicit feedback loop that requires self-evaluation on a specific 
time" (Zimmerman & Moylan, 2009, p. 302). Thus, this lack of realistic deadlines 
evidenced in both cases may become an obstacle for learners when attempting to 
fulfil goals focused on learning and skills development in self-paced MOOCs. 

This multiple case study suits the COVID-19 era, which has led most learners 
into an open world learning where they are anticipated to employ effective strate- 
gies to sustain their education at a distance. The findings have indicated that lan- 
guage learners can engage in goal setting processes when studying beyond the 
classroom in an LMOOC or a specific content-based MOOC, at least within the 
context of this study. Participants formulated outcomes they wanted to attain while 
learning independently in such online courses. However, they did not include 
time-bound goals, which raises questions about the realistic and achievable nature 
of the outcomes they set for their MOOC-based learning. The online learning 
experience described in Chapter 5 can count as the first steps towards SRL. Yet, 
learners still need to overcome various obstacles, such as setting vague and unreal- 
istic goals, to initiate their self-regulation in the face of adversity. 


5.6.1 Implications for practice 


It is crucial for educators and learners to identify how learners set and assess their 
goals when learning independently. Researchers and educators need to understand 
and facilitate goal-setting processes beyond the classroom, especially the self- 
evaluation of outcomes on a specific time. Educators can take advantage of the 
MOOC?’ potential for independent learning to encourage learners to initiate their 
self-regulated learning. Clear guidelines on setting specific, realistic, measurable, 
and attainable goals should be incorporated into those initiatives so that learners 
can effectively self-regulate their learning in an open world. 
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Chapter 6 


Discourse practices in MOOC 
discussions 


A corpus linguistic approach 


Shi Min Chua 


6.1 Introduction 


In massive open online courses (MOOCs) the discussion function allows learners 
to exchange information, experience and ideas with each other (Ferguson & 
Sharples, 2014; Poquet, Dowell, Brooks, & Dawson, 2018). The discussion has been 
hailed as beneficial for socio-constructive learning (Sharples & Ferguson, 2019). In 
a socio-constructive process, humans co-construct meaning, knowledge and the 
social world through social interactions with others (Vygotsky, 1978). Language is 
one means for such co-construction. However, limited MOOC research has 
investigated learners’ language practices in online discussions. Instead, most studies 
have focused on evaluating learners! comments in the discussion, such as whether a 
comment showed a learner engaged in critical thinking or a comment was on-topic 
(Kellogg, Booth, & Oliver, 2014; Wen, Yang, & Rosé, 2014; Wise, Cui, & Vytasek, 
2016). Other MOOC studies in the field of educational technology also centred on 
the implications for learning and technological design (Almatrafi & Johri, 2019), as 
also illustrated in Chapters 5, 8, and 9 (Conde Gafaro, 2022; Iniesto, McAndrew, 
Minocha, & Coughlan, 2022; Rizvi, Rienties, Kizilcec, & Rogaten, 2022). 

As will be elaborated in the next section, the analysis of learners’ comments in 
MOOC discussions in previous studies was mainly quantitative in nature 
(Kellogg et al., 2014; Wen et al., 2014; Wise et al., 2016). However, the richness 
of the textual data in the learners’ comments was often lost in the quantification 
process of these analyses, such that learners’ discourse practices in the socio- 
constructive processes were not revealed. Therefore, there is a lack of 
understanding of learners’ discourse practices in initiating and engaging in 
conversations with others in the MOOC discussions. Discourse practices refer 
to the recurring ways a community uses language to do things in their social 
context — above and beyond a sentence (Herring, 2004). To fill this gap in 
MOOC research, I conducted a corpus linguistic analysis of MOOC discussions 
in my PhD to explore discourse practices that facilitated and hampered 
conversations among learners. In this chapter, I will introduce this methodology 
and summarize the main findings from my PhD to illustrate the importance of 
language practices in MOOC discussions. 
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6.2 Previous research on MOOC online discussions 


To the best of my knowledge of the literature and based on the systematic review 
by Almatrafi & Johri (2019), most studies on MOOC discussions used a coding 
and counting paradigm of content analysis to analyse learners’ comments either 
manually or automatically by machine-learning techniques. The comments typi- 
cally were analysed individually, rather than discursively to understand how a 
conversation was co-constructed by learners. These studies categorised learners’ 
comments — for example, whether a comment was on-topic or off-topic (Wise 
et al., 2016), whether a comment indicated that a learner engaged in higher order 
thinking or paid attention (Wang et al., 2016) or whether a comment was positive 
or negative (Wen et al., 2014) — rather than how a comment was written, for 
example, how a question or a disagreement was raised to engage with others. 

In these studies, learners' textual contributions were reduced to codes for count- 
ing purposes such that their comments in the discussion could be quantified and 
(cor)related to other variables statistically, such as learners’ learning outcome, partici- 
pation pattern, or course design, as, for example, done by Rizvi et al. (2022). The 
coding also formed the basis for automatic recommendation of quality comments 
to learners, monitoring of the discussion space, and prediction of learning perfor- 
mance (Almatrafi & Johri, 2019; Wise et al., 2016). These studies were useful for 
evaluating the quality of the discussion, and might inform educators or designers for 
further improvement of their MOOCs. 

However, equally important are the socio-constructive processes that are realised 
by learners' actual language and discourse practices (Vygotsky, 1978; Wegerif & 
Mercer, 1997; Wise & Paulus, 2016). Several researchers have argued that learners 
need to be aware of discourse practices that are suitable for online communica- 
tions, especially for negotiation in online discussions (Herring, 2004). For example, 
in a small-scale online learning discussion, sharing of experience could be a way to 
reach agreement and affiliation in co-construction, or could be rejected by others 
as authoritative, depending on how learners oriented it in the ongoing discussion 
(Kaanta & Lehtinen, 2016). In another online learning discussion, Littleton & 
Whitelock (2005) found that in the process of socio-constructive learning, learners 
did not necessarily employ reasoning but could express uncertainty instead, for 
example, “Just some ideas which may or may not be of help”. A learner also 
responded to another learner’s question by asking another question to give a hint 
to the solution rather than giving away the answer. 

As shown in both studies, the way how an experience or idea was shared could 
impact the dynamic of the ongoing conversations as well as the social relationships 
among learners. These findings, although not based on MOOCs, highlight the 
importance of investigating the comments discursively within discussion threads as 
well as examining the role language plays in socio-constructive processes. 


6.3 A corpus linguistic approach to MOOC discussions 


To harness both the big data available from MOOC online discussions and the rich 
language data to explore discourse practices employed by learners, Chapter 6 
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introduces a mixed methodology, corpus linguistics, to MOOC research. This is 
also an attempt to promote linguistic perspectives and methodology in the field of 
education where textual data are investigated. This mixed methodology provides 
quantitative evidence without reducing the textual data into codes, unlike the 
content analysis used in previous MOOC research. 

A corpus linguistic approach consists of a set of established procedures and 
methods, including keywords analysis, concordancing, and collocation analysis (to 
be introduced in the following subsections), that can be used in combination to 
investigate language use and patterns in large bodies of textual data (McEnery & 
Hardie, 2012). All these methods are based on the assumption that with a corpus of 
a suitable size, recurring (and rare but important) language patterns can be identi- 
fied and analysed to reveal language usage, discourse practices and language users’ 
construal of the social world and their interactions. 


6.3.1 Keyword analysis 


Keywords refer to words used statistically significantly more often in the corpus 
when compared to another corpus, thus suggesting the “aboutness” and “styles” of 
the corpus under investigation (Baker, 2004). A word is considered a keyword 
when the p-value for the log-likelihood ratio test is p-value « .000000000001 
when comparing to another corpus (Flowerdew, 2008). The keywords found can 
then be subjected to more in-depth discourse analysis to understand how the key- 
words are used in discourse practices. In short, keyword analysis is a corpus-driven, 
or data-driven approach that starts from quantitative analysis, then moves onto 
qualitative analysis for interpretation. 


6.3.2 Concordancing 


Concordance lines show the word of interest in their co-text, i.e., a span of char- 
acters or words, in a vertical format, as shown in Figure 6.1 which is a display from 
corpus tool Antconc (Anthony, 2017). The usage and senses of the word of interest 
across the corpus could thus be analysed qualitatively (Sinclair, 2003). It can be 
considered as the main method for qualitative analysis in corpus linguistics 
(McEnery & Hardie, 2012). The concordance lines facilitate the observation of 
recurrent language patterns around the keyword or word of interest. It is based on 
the assumption that the meaning or social function ofa word is contributed by the 
other words regularly co-occurring with it. 


6.3.3 Collocation analysis 


Similar to concordancing, collocation analysis is used to examine recurrent lan- 
guage patterns. Collocates refer to words co-occurring often with the word of 
interest or keyword, typically within a five-word window preceding or following 
the word. An effect size measure, mutual information 3 (MI°) is used to indicate 
how much the observed co-occurrence frequency of the two words exceed 
expected frequency (McEnery & Hardie, 2012). 
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Vitally important. An example of how it went wrong........ A ago a company (I had 
the ability to " "sense" " were the students goes (a goc can diagnose what the problem 
others result, the time to get there looks A - Story, sources from NASA: On 
my own spoken corpus. What am | doing ?"A talk, but | do wish 
someone who refuses to believe that they are about E is human nature. | am 
round. please, let me know if | am about zt pandas.io.wb import download 
lot and | started to remember all the actions in the classroom. | was 
do not realise when they have spelled them Ad -term memory issues and spelling | 
4 bit windows. | assume there is something simple All same answer, good-o! Did 
can't be correct. What have i done ? All received! Thanks" "I may be 
s that grammatically make sense, but seem somehow all . i.e. they are not 
as Lapetus or have | been getting it all ? | notice that one of 
the pages or their overwhelming content. Am I ? Although was all wrong the operation 
and reversals of number e.g., withdrawing the amount from an ABM. Tom might 
or fibers. | believe the " "diet" " approach is .An " "what nutrients we should tak 
be so useful in that context. Am I ? 56 and day transit + fuel. 9 days on 
stand health messages and are health literate are ,an confused by conflicting "evidence" 
amount of incredibly difficult reading texts, am | ? And work in a franchising language 
would like to understand what I'm doing an it in a healthy way" 
can give warnings where things may be going and give the opportunity to step in 
was childbirth (not infectious disease). Is this ? and we know? Also: no, no 
to change so I didn't get them an to hold down my notebook 
right but a right as another hence an dock points the answers should 


Figure 6.1 Concordance lines of wrong in my corpus. 


These three procedures reveal repeated patterns of language usage in the cor- 
pus, thus facilitating the observation of discourse practices that are common in 
the language community. Both concordance reading and collocates can be used 
to further investigate how a keyword is used in a corpus and realises discourse 
practices. 


6.3.4 Quantitative and qualitative analysis in corpus 
linguistics 


A corpus analysis typically involves both quantitative analysis and qualitative analy- 
sis (Biber, Johansson, Leech, Conrad, & Finegan, 1999; McEnery & Hardie, 2012). 
Quantitative analysis is conducted on the frequency data, i.e., the number of occur- 
rences of words or linguistic patterns in the corpus, typically achieved by keyword 
analysis, frequency count, and collocation analysis. The qualitative analysis is con- 
ducted on the co-text or context where a word occurs in the corpus, typically 
achieved by concordancing (McEnery & Hardie, 2012). Both analyses are usually 
conducted in synergy to examine the form and function of language use (Biber 
et al., 1999). Forms, that is words, or collocates, are the basis of quantitative analysis, 
whereas function is examined by the qualitative analysis. Usually, some kind of 
discourse analysis is conducted for the qualitative analysis alongside concordancing 
in corpus analysis (Baker, 2004). 


6.4 Previous corpus analysis on MOOCs 


A brief corpus analysis was conducted on a MOOC “How to read your boss" by a 
corpus linguist (Collins, 2019). Collins employed keyword analysis, concordancing 
and collocation analysis to investigate learners' use ofa technical term taught in the 
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MOOC, face, in their comments to explore evidence of their learning. In his corpus, 
face was found a keyword that was used significantly more often when compared 
with British National Corpus of written English (BNC, Leech, Rayson, & Wilson, 
2001). However, it was used mainly when learners responded to the discussion 
prompt in the course materials, “Is ‘personal face’ or ‘social identity face’ more 
appropriate for your workplace?". This was evidenced by a collocate of face, social 
identity, which also appeared in the discussion prompt. Collins (2019) found that this 
technical term was seldom used in other contexts, suggesting that learners might 
not have acquired the concept of the term to generalise it to other contexts. 
Nonetheless, Collins’ concordancing of the keyword face showed that learners might 
engage in the socio-constructive process of learning in MOOC discussions. For 
example, learners explicitly invited others “Would somebody explain the difference 
between Personal Face and Social Face?" (p. 142) or expressed their uncertainty “I 
don’t feel I fully understand the difference between personal and social face" 
(p. 142). However, he did not further explore this aspect of learners' discourse. 
Collins (2019) also investigated keywords in posts and replies to explore interac- 
tivity in the discussions, by comparing each type of comments to BNC. Keywords 
found in the posts included identity, boss, personal, I, am whereas keywords found in 
the replies included hi, I agree, you, I, am. However, he did not explore discourse 
practices realised by these keywords. That is, he stopped at the quantitative analysis 
of keyword analysis. Collins (2019) himself suggested more in-depth discourse 
analysis was needed to understand learners’ conversations in MOOC discussions. 
Although Collins' study focused only on one concept taught in the MOOC, he 
successfully showed that, besides the often-used coding and counting paradigm, a 
corpus linguistic approach could reveal textual evidence of users’ learning and 
interactions in MOOCs. He showed that educators could use the corpus methods 
to examine learners’ learning in the MOOCs they teach. He also showed a pre- 
liminary finding that discourse in posts and replies were different, although how 
learners employed the discourse to engage in conversations remained unexplored. 


6.5 Present study: a large-scale corpus analysis of MOOC 
discussions 


Building on Collins! study (2019), I conducted a large-scale corpus analysis of 
MOOC discussions in my PhD to explore discourse practices that facilitate or 
hinder socio-constructive processes of learning. My corpus consisted of 
11-million-word learners’ contributions (202,787 comments) in 12 FutureLearn 
MOOC-s. For information about FutureLearn, the compilation of the corpus and 
ethical considerations of analysing learners! comments, readers are referred to my 
PhD thesis (Chua, 2020). 

In my PhD, I asked the question of how learners initiate and engage with each 
other in MOOC discussions. To answer this question, I conducted two keyword 
analyses: (1) comparing posts that receive replies, i.e., initiating posts that initiate a 
conversation, to those that do not, i.e., independent posts which are not part of a 
conversation; (2) comparing replies to these two types of posts. The first keyword 
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analysis aimed to reveal discourse practices that are more likely to initiate a conver- 
sation, while the second keyword analysis to reveal how learners respond and 
engage with others. The corpus analysis thus provided insights to the beginning 
and sustaining of the socio-constructive process among learners. 

The analysis found 69 keywords of initiating posts (initiating keywords), 77 key- 
words of independent posts (independent keywords) and 57 keywords of replies 
(reply keywords). I then conducted concordancing and collocation analysis of each 
type of keywords to examine the discourse practices realised by these keywords. It 
is not possible to elaborate and present statistics for every single keyword in Chapter 
6, so in the subsequent sections, only selected findings are summarized, with key- 
words italicised. 


6.6 Findings 
6.6.1 Discourse practices in initiating posts 


Different discourse practices were employed in initiating posts and independent 
posts. Firstly, compared to independent posts, learners tended to use more modals 
(might, would, could) and hedges (perhaps, seems, sort of) to soften their claim (e.g., This 
could perhaps mean that British fiction, in comparison with American fiction, uses 
more dialogue?) in initiating posts. This softening was also expressed by indicating 
their uncertainty (I wonder, I am wondering), not knowing (I don’t know), possible 
mistake (I might be wrong, am I missing something?) or if-conditionals. This softening 
or qualification of one’s claim ensured that their posts did not come off as bare 
assertions or authoritative that did not allow alternative voices (Littleton & 
Whitelock, 2005; Martin & White, 2005). Rather, these initiating posts framed in 
tentativeness created room for others to pitch in, thereby increasing the chance of 
receiving replies and beginning a socio-constructive process with others. 

Secondly, in the initiating posts, although learners did not know who was going 
to reply them, they established a dialogue with potential conversational partners 
with anybody and anyone (e.g., does anybody have a good suggestion; Anyone heard 
of Hildegard von Bingen’s contributions to plantlore?). This framing indicated 
their invitation to any learners in the MOOC to join their conversations. Besides, 
these two initiating keywords were also used to seek shared experience or prob- 
lems (e.g., Anybody else not counting calories?; Anyone ever had HAD to use the 
old fashioned earth toilet). This framing suggests that learners co-constructed their 
experience in the discussions. Additionally, learners addressed other users with any- 
one and anybody, instead of addressing only facilitators, suggesting that learners were 
aware of the socio-constructive learning function of MOOCs, i.e., learning via 
conversation with each other, instead of an educator-centred transmission model 
(Sharples & Ferguson, 2019). 

Thirdly, meta-language, that is language used to explicitly refer to learning or 
discussion (article, question, example) were also used more frequently in initiating 
posts. Using these words in their posts to highlight what they referred to (e.g., 
Question: If climate change is; That was quite hard for me to comprehend the 
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article; A fine example can be found) provided a concrete common ground for 
others to reply to. 

In short, discourse practices that indicated tentativeness addressed potential con- 
versation partners, referred explicitly to a common ground seemed to increase the 
chance of receiving replies and initiating a conversation. The discourse practices 
facilitated socio-constructive learning by welcoming alternative voices or shared 
experiences pitched by others, and explicitly referring to issues to be discussed. 


6.6.2 Discourse practices in independent posts 


Learners used fewer initiating keywords and the discourse practices explained 
above in the independent posts. This might explain why independent posts did not 
receive replies. In contrast to addressing other learners, in independent posts, learn- 
ers engaged in self-references (I, my, our) and expressed appreciation (e.g., Excellent 
range of resources, t/ianks!). Instead of stating their uncertainty, learners tended to 
express their learning goals (e.g., I’m really looking forward to learn) or outcomes 
(e.g., I enjoyed this course and definitely learned a lot). These expressions of self- 
references, appreciation and learning goals and outcomes could be prompted by 
the learning activities or content at the start (e.g., Describe your interest in ... and 
... tell us what you hope to get from this course) and at the end of each MOOC 
(e.g., What have you found to be good, useful or interesting during this course?). 
Since learners were responding to the prompts, the posts might not be written to 
engage with others. Learners’ own reflections with the learning activities and con- 
tent also played an important role in learning (Ferguson & Sharples, 2014). 

Learners also voiced their opinions in independent posts, as evidenced by the 
independent keywords think and agree (e.g., I think we should be open to the possi- 
bility; I agree ... so it enables...). However, nobody joined the conversations to 
exchange opinions, so the socio-constructive process of engaging with each other 
did not happen. There were times that learners merely expressed their agreement 
without further elaboration. Although these agreements indicated their presence in 
the discussions and created a positive environment, the lack of additional substance 
in their independent posts might explain why nobody replied to them. 

More importantly, most of the independent posts with phrases I think and I agree 
seemed to be in response to learning activities with prompts “Do you think” and 
“Do you agree”. The framing of the prompts might have primed learners to self- 
reference and respond to the questions on the page, thus very few learners 
responded to each other to co-construct their ideas. Admittedly, it is fairly com- 
mon that internet users respond to content on the page more often than engaging 
with each other (Herring, 2013). Therefore, the occurrence of independent posts 
in the discussion was not solely due to the framing of the learning activities. 

In short, the discourse practices in the independent posts were mainly apprecia- 
tion of the MOOC, reflection of learning journey, or voicing of opinions, all of 
which seemed to be addressed towards the prompts of the learning activities. This 
might explain why these posts did not receive any reply since these discourse prac- 
tices did not create room for others to pitch in. 
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6.6.3 Discourse practices in replies 


9196 of the initiating posts received between one and four replies, suggesting that 
most conversations among learners in the discussions were relatively short-lived, 
and might not be conducive for socio-constructive processes. Most replies were 
expressions of agreement (yes, agree, agreed, true, right, exactly, absolutely, totally, indeed) 
and expression of appreciation (thanks, thank). The expression of agreement, espe- 
cially in short replies, might not generate a conversation. However, it was a way for 
learners to indicate their engagement with the posts, as shown in the short thread 
below. The expression of agreement could be considered a socio-constructive pro- 
cess of shared experienced. 


INITIATING POST: It's interesting that the issues Dyslexic students may 
extend into aural aspects of language...... I like the idea of focusing on recep- 
tive subskills...... I think we often make the mistake of focusing on product 
over process in learning, and it seems to me that for Dyslexic students, a focus 
on process is absolutely essential. 


REPLY |: Yes — I'm guilty of this. 
REPLY 2: I would strongly agree with that! 


Although the reply keyword agree and agreed were mainly used for expressing 
agreement, the concordance reading of agree revealed that users employed the 
phrase agree to disagree/differ in replies, especially in long discussion threads. The 
phrase was used to end their discussions when they could not reach agreement 
after voicing their disagreement. In these conversations, although they did not 
manage to co-construct a final verdict, they were at least exposed to different 
views, suggesting socio-constructive learning. 

Another discourse practice found in replies was indication of tentativeness, similar 
to the discourse practices in initiating posts. However, it was realised with hedges 
maybe and probably, which are used more often in oral language, suggesting the dia- 
logic nature of replies (Biber et al., 1999). The function of these two hedges in a 
socio-constructive process is best illustrated in one discussion where a few learners 
employed these two hedges to provide advices to a learner who posted about difficul- 
ties in providing their stepchildren a healthy diet. Two of the replies were shown here. 
The numbering of the replies indicated its order within the discussion thread. 


REPLY 3: ...... The idea of a family meal in your case sounds pretty hellish but, 
maybe, there are other things going on in their minds and it is not just 


REPLY 9: It must be soul-destroying for you, but you are probably wise not to let 
meals become more of a battleground...... 


In these two replies, the hedges made the advice-giving less directive and 
authoritative. This was evidenced by other replies by learners contributing these 
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two replies, “Hope that doesn’t sound awfully patronising!” “sorry if it sounds sim- 
plistic”. The learner who asked for advice also took on the advice by saying “Thank 
you everybody for your comments, I am agreeing and...... ”. The hedges might 
have created room for the learner to respond to the advice and explain their situ- 
ations more. The advice-giving and learner’s further explanation in response to the 
advice suggested that learners engaged in a socio-constructive process. 

Another discourse practice found in the replies was meta-language referring to 
each other’s comments (reply, posting, comment, post, point, said, say, you, your), suggest- 
ing that learners engaged with others’ comments. The meta-language could be a 
double-edged sword. On one hand, it was used to clarify each other’s comments, 
and facilitate resolution of misunderstanding, as shown in the following exchange 
between two learners. 


REPLY 4: I did not say it is a waste but it seems to be a waste if you dispose of it 
without taking any benefits from it..... 

REPLY 5: sorry, I thought you said supplements with added protein are clearly a 
money making scheme. It does make you think though, which is the point of 
the course..... 


On the other hand, using meta-language to question others’ comments without 
acknowledging their clarification could lead to stalemate in a discussion, hindering 
the socio-constructive process. The exchange below illustrates this situation, which 
came from a discussion thread where these two learners repeatedly criticized each 
other’s comments. 


REPLY 4: ...... In another post, about apple cider vinegar, you specifically say "Two 
words - Take It In this post you tell us ‘trust me, it works’. That sounds to me 
as if you are recommending it to anyone who reads that comment. ... .. 

REPLY 5: ...... It seems as though you consider my phrasing to have been flippant but 
you don't have to post long, in-depth comments on everything...... 


In short, learners’ discourse in replies indicated their engagement with others’ 
comments, although most discussions were short-lived. Expressions of agreement 
and appreciations created a positive space in the discussions, while framing responses 
with tentativeness facilitated co-constructions of solutions. Meta-language seemed 
to be useful to clarify misunderstanding for socio-constructive learning, but over- 
using it could lead to stalemate. 

It is worth noting that a keyword analysis comparing replies in long conversations 
(more than five replies) versus short conversations (fewer than five replies) found no 
keywords in long conversations, suggesting either that there might not be any word 
or expression that could increase the chance of sustaining a thread, or that discourse 
practices sustaining a thread were not realized by particular keywords or expressions. 
Therefore, an in-depth discourse analysis was conducted to further examine long 
threads in my PhD, which will be presented in future publications. 
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6.6.4 Unexpected reply keyword: link 


As mentioned earlier, keyword analysis is a data-driven approach that could unravel 
unexpected observations in the data. One such observation was the reply keyword, 
link, which was used by learners to refer to URLs. Based on concordance reading 
and collocation analysis, learners typically appreciated others posting link(s) in the 
discussions (see Figure 6.2). However, very seldom did the URLs posted generate 
a discussion, and learners sometimes did not write much about the URLs posted, 
as shown below. 


Eat it. Much better for you than seed oils and polyunsaturates. https:// www. 
ncbi.nlm.nih.gov/pubmed/24723079 


There were a few rare cases where learners discussed extensively about URLs 
posted, suggesting a socio-constructive process. This happened when the learners 
held on opposing views and each posted URLs to support their own views. 
Although it is encouraging that learners critiqued each other's posted URLs, they 
seldom discussed the content linked to the URLs but the credentials of the 
authors. Most of the time, they remained unconvinced by each others posted 
URLs and arguments. Learners may need to be prompted to move beyond the 
URL itself but to discuss the content linked to the URL to facilitate the socio- 
constructive process. 


6.7 Discussion and moving forward 


MOOC discussion is a space where learners engage with each other to exchange 
information and ideas. Commenting in the online discussions differ from oral or 
written language, such that learners need to be equipped with language skills suit- 
able for communicating and learning in this new medium (Herring, 2013; 
Littleton & Whitelock, 2005). However, previous MOOC research has not yet 


only managed to find her interview in Guardian. Thanks Michele, for the link to exercises on concordances. 

nts and research funds. Very interesting Julian, thanks for that link @Carole Pope. Hi Carole, | 
the link. Thanks for the link! Very helpful! "Thanks for the link! When I live on Mars 
aters in the maria adn craters within craters. " Thanks for the stellarium link :) "Do you have a 
eiYzRfd2taTFplQOU/BEO6 AntConc compat.txt" Thanks for he link Simon. The by-products of 
that comes up Three days behind ya, Andy! Thanks for this link. Are these telescopes in orbit 
y+23%2C+2008 very useful link Thanks for the link Phil, very interesting. Many thanks for the link 
which confirm that the view noted in the link has been superceded. —" Many thanks David for your 
I think that they will be to you." Thanks for the MMM link, Carol. I'm already 
to funding the wars etc that Mike mentions." Thanks Mark, great link shows just how bad 'common 
have a look at your account of spelling: thanks for sharing the link. "Thank you for the 
't work Jonathan, that is such a useful link! Thank you - Thanks for your comments and the 
soil horizon photo with helpful measurements. Thanks for the link to Soil Science page. (I' 


Figure 6.2 Concordance lines of link in my corpus. 


86 Shi Min Chua 


paid attention to the language use in the discussion. By conducting a corpus analy- 
sis of 12 MOOC Futurelearn courses, Chapter 6 successfully unveiled useful dis- 
course practices that learners can employ to initiate and engage in conversations 
with others in MOOC discussions. The analysis showed that there were indeed 
differences between initiating posts that receive replies, independent posts that do 
not receive replies, and replies. 

Chapter 6 illustrated that a corpus linguistic approach is useful for examining big 
language data available from MOOC discussions while enabling discourse analysis 
of actual language usage at the same time. This mixed methodology revealed both 
the general patterns and specific discourse practices in the discussion. The quantita- 
tive component of keyword analysis or collocation analysis pointed to a specific 
keyword or collocate which discourse functions were examined qualitatively by 
concordancing. 

More importantly, this integration of quantitative and qualitative analysis can 
guard against the possible presumption of the researchers regarding the discourse 
functions of certain words. For example, the keyword wrong did not necessarily 
carry a negative meaning but was used typically by learners to express their 
uncertainty, whereas the keyword link was not used to establish logical reasoning, 
but to refer to a URL posted. Both keywords wrong and link might be coded 
otherwise in coding and counting paradigm in other MOOC research, which 
might prevent us from understanding learners’ actual discourse. Furthermore, 
current machine-learning techniques used in MOOC research (e.g., Wise et al., 
2016) and discussed in Chapter 13 (Hillaire, Rienties, Fenton-O'Creevy, Zdrahal, 
& Tempelaar, 2022) typically discard function words, such as modals, grammatical 
words, and pronouns which have been well-established as linguistic resources for 
social relationships (Biber et al., 1999). As shown in Chapter 6, some modals were 
used to initiate conversions with others. This shows that a corpus linguistic 
approach may have much to offer for exploration of discourse practices for 
socio-constructive processes. 


6.7.1 Implications for practice 


Based on the findings in Chapter 6, several writing tips for online discussions can 
be provided for learners and facilitators: 


1 Do not make sweeping generalizations and bare assertions. Rather, use modals, 
hedges, if-conditionals to soften or qualify your claims, such that others' alter- 
native views are not rejected. 

2 Express uncertainty and tentativeness in your claims, such that others will be 
more willing to fill in the gap. 

3 Be explicit in referring to the issue or topic you are writing about, such that 
a common ground can be established. 

4 Use oral language to address potential audience, and seek others who share 
similar experiences or problems. 

5 Recognize others’ viewpoints and acknowledge what others have written. 
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6 You can agree to disagree, rather than having a “winner” in a debate. 

7 It is ok to ask for clarification from others regarding their comments, but do 
not repeatedly criticize their ways of posting. 

8 Write about the relevance of the URLs to the current discussions. 


The findings in Chapter 6 could also inform design of the discussion prompts. 
Although the prompts “Do you think" and “Do you agree” typically introduce a 
question that requires learners to voice their opinions, this framing inevitably prompt 
learners to post rather than replying to others. Perhaps additional prompts such as 
“How do you find others’ view in relation to yours?" can be added. At the same time, 
the number of questions in the prompts can be reduced, such that learners have a 
focused common ground to engage with each other. Nonetheless, the effectiveness 
of this suggestion awaits to be examined. It is also worth reiterating that online users 
tend to post in response to content on the page than respond to others. Therefore, it 
1s unavoidable learners do not necessarily reply to others but the prompts. 
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Chapter 7 


Simplification of open educational 
resources in English 


Its effect on text processing of English learners 


Irina Rets, Ursula Stickler, Tim Coughlan and 
Lluisa Astruc 


7.1 Introduction 


Open Educational Resources (OERs) are an increasingly important part of the 
contemporary provision of education. Discussions about OERs are generating 
substantial interest regarding how these resources can reduce educational inequal- 
ity, and decrease the cost of education, particularly in developing countries (e.g., 
Cobo, 2013). At the same time, a number of concerns have been raised about 
OERS failing to widen access to education (Casserly & DeBarger, 2020; Papathoma 
et al., 2020). Despite the aspirations to fundamentally open up education, OERs 
are still mainly used by well-educated learners residing in the Global North, and 
most OERs are offered in English (Farrow, de los Arcos, Pitt, & Weller, 2015). 

A recent study by Rets, Coughlan, Stickler and Astruc (2020), which examined 
text complexity of 200 OER reading materials across different educational levels 
and subject categories from two major OER platforms, provided some empirical 
evidence supporting these concerns. The study showed that more than 86% of the 
examined OERs require an advanced level of English language proficiency. Thus, 
there might be a gap between many potential OER learners' language abilities and 
OERs that are expected to enable inclusive education. As a number of studies 
showed that one size does not fit all, particularly in online education, which gives 
immense opportunities for a personalised learning (e.g., Rets, Rienties & Lewis, 
2020; Rienties, Lewis, O'Dowd, Rets & Rogaten, 2020), it is important to evaluate 
solutions that can make OERs more accessible globally. 

Despite the scepticism of open education to help learners from non-English- 
speaking backgrounds, there is a lack of OER studies that conceptualise and test 
solutions for improving the linguistic accessibility of OERs to this global audience. 
Some solutions described in these OER studies focus on how to customise OERs 
to specific national contexts, such as translating OERs into local languages. Yet, 
such approaches do not generalise to a wider learning context (Casserly & 
DeBarger, 2020). 

One solution that can potentially increase the linguistic accessibility of OERs is 
text simplification. Text simplification is the process of modifying authentic texts, 
or texts written for native speakers of a given language, with the intent to reduce 
the language level of these texts and increase their accessibility for the non-native 
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speakers of this language (Tickoo, 1993). Previous studies on text simplification 
showed that simplified materials can enhance the learners comprehension of the 
text, increase learner autonomy, and provide more opportunities for a learning suc- 
cess scenario (e.g., Crossley & McNamara, 2016). 

At the same time, several questions arise from the existing text simplification 
research. First, empirical evidence in previous text simplification studies has been 
obtained using traditional methods of reading research, such as comprehension 
tests, which might limit the breadth and depth of the analysis. Since reading is a 
cognitive activity that involves lower- and higher-level processes, there is a need 
to also explore the “processes of reading” rather than only investigate the “product 
of reading”, which is text comprehension (e.g., Brunfaut & McCray, 2015). 
Secondly, no previous research on text simplification has been conducted in the 
OER context, despite the calls for more accessible OERs. With this in mind, the 
primary goal of this chapter was to obtain emerging evidence on the effect of 
OER text simplification on text processing of non-native English speakers 
(NNES).This research primarily used eye-tracking stimulated recall methodology 
and was underpinned by Khalifa & Weir’s model of reading (2009), which are 
described next. 


7.2 Eye-tracking stimulated recalls to investigate text 
processing 


An increasing number of studies investigate text processing through the use of eye- 
tracking (Conklin, Pellicer-Sanchez & Carrol, 2018). In the field of reading 
research, eye-tracking is defined as the real-time registration of an individual's eye 
movements, typically as they read the information on a computer screen with an 
eye-tracking technology integrated or attached to it (Conklin et al., 2018). Eye- 
tracking is also used as a stimulated recall interview technique in reading research, 
as part of which the recorded eye movements of the reader are played back to them 
after the reading task in order to stimulate the thoughts they were having during 
reading (e.g., Brunfaut & McCray, 2015). 

Conceptualisations of processing levels vary depending on the focus of the 
underpinning reading theory. This research was mainly informed by Khalifa & 
Weirs model of reading (2009). This model was particularly relevant for this 
research due to its componential approach to researching text processing, which 
makes the model amenable to transformation into a research instrument to be used 
for data analysis and data coding purposes. The model comprises a hierarchical 
system of eight distinct cognitive processes, which are thought to tap into different 
levels of processing complexity and which by working together result in text com- 
prehension. These comprise the following: 


* so-called lower-level processes — word recognition, lexical access, syntactic 
parsing, and establishing propositional meaning; 

e higher-level processes — inferencing, building a mental model and creating a 
text level or intertextual representation. 
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Amongst the studies that explored text processing of NNES and were both 
informed by this model and used it as a coding framework for qualitative data 
analysis is Brunfaut and McCray’s work (2015). The study used eye-tracking stimu- 
lated recall interview data to describe the kind of text processing participants were 
engaged in during reading in language test conditions. The study showed that 
almost the entire range of cognitive processes, as specified by Khalifa and Weir 
(2009) (except for intertextual representation), was used by participants while 
completing the test reading component. This suggested that the test quite compre- 
hensively tapped into the construct of reading. Furthermore, the study found some 
processing trends associated with participants! language proficiency, such as rela- 
tively more frequent use of syntactic parsing and paragraph-level representations, 
but less frequent use of lexical access processing by more proficient participants. 

Chapter 7 aims to pilot a potential solution for increasing the linguistic acces- 
sibility of OERs to NNES by eye-tracking stimulated recall interviews. Under- 
pinned by Khalifa and Weirs model of reading (2009) this research allowed a 
comparison between the types of processing strategies verbalised by participants 
across the two text conditions — simplified OERs and authentic (unmodified) 
OERs. As such, the research question of this chapter was as follows: What is the 
effect of text simplification on text processing, as evidenced in the frequency of use 
of cognitive processing strategies by NNES in eye-tracking stimulated recall 
interviews? 


7.3 Materials and methods 
7.3.1 Participants 


Our aim was to recruit a sample that would reflect the diversity of the population of 
OER learners. Since OERs are developed as universally available educational resources 
(Cobo, 2013), OER learners constitute a diverse audience of learners regarding their 
educational background, age, and location. As the overall aim of this chapter was to 
explore how lower-level proficiency NNES respond to OER text simplification, only 
participants’ language proficiency was controlled during sampling. 

Twelve adult NNES took part in this research on a voluntary basis. Due to cali- 
bration problems and common problems with eye-tracking data quality (Catrysse, 
Gijbels & Donche, 2018), only data of nine participants (Mge = 37.6, SD = 5.41) 
were available for the analysis. All participants were female, which was a reflection 
of the population from which they were recruited and which was a predominantly 
female group. All participants were recruited from the same class, an intermediate 
(B1) English language course, at a local adult community learning centre in the 
UK. Their language level was determined by this education centre through the 
entrance language examination and was benchmarked against the Common 
European Framework of Reference for Languages (CEFR) (Council of Europe, 
2001). In terms of participants’ educational background, most participants were 
university graduates (n = 6), n = 2 had vocational degrees, n = 1 had an A-level 
qualification. Participants’ language backgrounds varied to reflect the diversity of 
the OER learner population generally. 
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7.3.2 Texts 


Two OER texts in the domain of natural sciences were selected from the OpenLearn 
(2020) platform: Text 1 (160 words, two paragraphs) was selected from the OER 
course “Why sustainable energy matters", and Text 2 (145 words, one paragraph) — 
from the OER course "Galaxies, stars and planets". Both selected texts were part of 
the first section of introductory courses; the courses required no prior educational 
background. To control for the learning effect in each reading, the selected texts 
represented different topics, but were within a largely similar topic domain. As there 
is no single approach to simplifying texts, text simplification in this research was 
performed in line with the text complexity categories revealed in the earlier works 
of the first author (Rets, Coughlan, et al., 2020; Rets & Rogaten, 2021). The text 
simplification strategies used in this chapter are presented in Table 7.1. 

The final version of simplified Text 1 contained 164 words, two paragraphs; 
simplified Text 2 contained 147 words, one paragraph. Thus, a total of four texts 
(two original texts and two simplified versions of these texts) were used. For fur- 
ther details of the formatting of the text, and the technical characteristics of the 
eye-tracking equipment, please, see Rets (2021, pp. 128—129). 


7.3.3 Procedure 


The session started with participants signing a consent form, completing a partici- 
pant background questionnaire, and receiving oral instructions for their reading 
task. It was explained to participants that in this research their text comprehension 
would not be tested. However, since reading is a purpose-driven process, and in 
line with Catrysse et al. (2018), they were asked to read the texts as if they were 
taking the final language examination at their language learning centre. Reading 
was self-paced, and participants were asked to indicate they finished reading each 


Table 7.1 Strategies used to simplify OERs 


Text simplification strategy Description of the strategy 

Sentence length Reduce the average number of words per 
sentence 

Word repetition Increase the proportion of repeated words 
in the text 

Word length Reduce the average number of syllables per 
word 

Noun elements per sentence Reduce the average number of noun 
elements per sentence 

Amount of elementary and Decrease the proportion of advanced lexis 

advanced lexis in terms of CEFR 

Word frequency Increase the proportion of commonly used 
lexis 

Logical connectives Increase the proportion of logical 


connectives between/within sentences 
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text by pressing the escape button on the keyboard. After receiving the task instruc- 
tions, a technical eye-tracking calibration test was conducted. This was followed by 
participants reading from the computer screen, while their eye movements were 
recorded. The texts were presented one at a time on the screen. Each participant 
read two texts: they first read either an authentic or a simplified OER, on one of 
the two topics outlined in Section 7.3.2. 

The reading of each text was immediately followed by a stimulated recall inter- 
view in English on participants’ cognitive processes during reading. The interviews 
were conducted using the gaze plot videos produced by the eye-tracking software. 
Before each interview, each gaze plot video was visually inspected to check the 
eye-tracking data quality. Eye-tracking data from three participants had a drift, and 
stimulated recalls were not conducted with these participants. 

In the gaze plot videos with the remaining nine participants, a moving red dot 
represented the point of fixation and the size of the dot was an indication of how 
long a fixation lasted. The replay was slowed down in order to give participants 
time to verbalise what they were thinking about during reading. The replay was 
paused after each fixation and a look-back (the times each participant looked back 
in the text). The research protocol with the interview script used in this research 
included such questions as: 


Here you fixated a lot / you are going back in the text. 
Why, do you think, you fixated on / looked back at this element in the text? 
What were you doing / thinking about? 


The stimulated recall interviews were recorded using a video camera to capture 
both the eye movement replay and participants’ verbalisations. The entire session 
with each participant lasted approximately 90 min. The visualisation of the research 
procedure is presented in Figure 7.1. 


START Introduction D 
Consent form; demographic 
questionnaire; task instructions J 


b d 
f Text 1 
Calibration; reading Text 1 with the 
eye movements recorded 


b. d 


Stimulated Recall Interview 
for Text 1 


i L_ | Counterbalanced 
Text 2 across the sample 


Calibration; reading Text 2 with the 
eye movements recorded J 


Ü 


Stimulated Recall Interview 
FINISH for Text 2 = 
Vv 


Figure 7.1 Visualisation of the data collection procedure. 
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7.3.4 Data analysis 


Our research question was concerned with the effect of OER text simplification 
on text processing of NINES using the qualitative evidence from the eye-tracking 
stimulated recalls. Data analysed were comprised of 18 stimulated recall interview 
sessions (n — 9 with participants reflecting on their reading of simplified OER. 
texts, and n — 9 — on their reading of the authentic texts). All 18 interview sessions 
were transcribed manually from the videotapes. The data were then manually 
coded in the qualitative analysis software package NVivol1, using the content 
analysis approach, as outlined by Neuendorf (2016). The aim was to analyse and 
code participants' thought processes during each eye fixation and look-back in the 
authentic and simplified texts they verbalised during the stimulated recalls. The 
average length of the transcribed interviews was 4000 words. 

In the first coding cycle deductive coding was employed, using Khalifa and 
Weirs (2009) model of cognitive processing in reading as the a priori coding 
scheme. The last level of the original model — creating an intertextual represen- 
tation — was removed from the coding scheme as participants read and reported on 
only one text at a time. In the second coding cycle inductive coding was employed 
to identify new processing strategies specific to the context of this research, which 
might not be reflected in the model of Khalifa and Weir’s (2009), since their model 
was primarily used for test validation. Three additional codes were arrived at during 
the inductive coding process. In line with Neuendorf (2016), two inter-rater reli- 
ability sessions were conducted to finalise the coding scheme. The percent agree- 
ment after the first inter-rater reliability session was 75%. Having revised the coding 
scheme, paying particular attention to the category descriptions, the second inter- 
rater reliability session was conducted with a different independent rater. The final 
coding agreement with the third rater was 90%. Altogether, 80 codes were identi- 
fied, which were then assigned to one of the ten cognitive processing strategies 
featured in the final version of the coding scheme. The final coding scheme used in 
this research with example quotes for each category is presented in Table 7.2. 


7.4 Results 
7.4.1 Cognitive processing strategy use across the sample 


The first two strategies that concerned lower-level processing, namely word recog- 
nition and lexical access, were featured in participants' verbalisations when partici- 
pants gave an account as to why they focused on a particular word in the text. Such 
accounts were mostly linked to participants experiencing confusion or difficulty in 
understanding the meaning of single words they encountered. Word recognition 
strategy seemed to be in use when participants tried to say the words out loud to 
themselves that they did not immediately recognise during reading. 


Participant 8: I don't think I’ve seen the word “current” before. I wasn’t sure 
how to pronounce it. Usually pronouncing the word to myself helps me iden- 
tify what kind of word this is and keep this word in my mind during reading. 
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Table 7.2 Final version of the coding scheme used in the study with example 


quotes 


Level of 
processing 


Category 


Example quote 


Lower level word recognition 
lexical access 
syntactic parsing 
establishing 
propositional 
meaning 
emotional resonance 
mother tongue 


interference 


vocabulary and 
grammar learning 


Higher level inferencing 


building a mental 
model 


creating text level 
representation 


“When | saw ‘encompasses’, | tried 
to pronounce it properly”. 

"Nevertheless...| was thinking to 
myself, is it ‘unless’ or something?" 

"| didn't know the previous word, 
that's why | slowed down". 

"| think | didn't understand this at 
all: replicated? On Earth? In 
Laboratories? Aren't we talking 
about the Universe?" 

“Thousands”, “billions” — “Oh, that's 
a lot of stars, | thought to myself”. 

“1970s” — “I said the year to myself 
in my language and then | realised 
| need to speak to myself in 
English!" 

“The Universe" — “we just learnt 
when to use 'the' in our English 
lesson yesterday, and | wanted to 
see how it is used in this context". 

"When | saw the word ‘atom’ - | 
remembered a line from my 
physics book in high school:'as 
small as an atom’”. 

“When | was nearly to the end of 
the text, | tried to understand 
everything before | finish the 
text". 

"Having finished reading the text, | 
reread the title — ‘Where do we 
get our energy from?’ ‘We’ — | 
established the reference with ‘we 
as humans’, and then | reread the 
first sentence and the last two 
sentences of the text to 
remember its key points”. 


The evidence that participants used lower-level processing strategies when 


struggling with the meaning of single words in the text was particularly salient 


when analysing the lexical access processing strategy. Lexical access was featured in 


participants’ verbalisations when they reflected on the reasons for their long fixa- 


tions on certain words in the text, talked about not knowing the meaning of those 


words, and trying to compare those words in their mind with the words they 


already knew that looked similar: 
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Participant 3: I confused the word "scarce" with "scary". I think I looked at 
this word longer because I don't know the meaning of this word. 


The other two strategies that constituted lower-level processing — syntactic parsing 
and establishing propositional meaning — were also used in the instances when 
participants could not understand the meaning of single words in the text. However, 
these strategies were used beyond fixating on only those single words and con- 
cerned fixating on larger lexical chunks in the sentence, such as word collocations 
and clauses. Both strategies were mostly associated with look-backs in the sentence. 
Syntactic parsing was used when participants tried to associate the meaning of a 
single word by looking back at a few surrounding words because they formed a 
conceptual unit together. Establishing propositional meaning was reported to be 
used when participants were looking back at larger context within a single 
sentence. 


Participant 4: Maybe I focused on the word “reserves” and then looked back 
at a couple of preceding words because I tried to read these two or three 
words together rather than understand them separately [when talking about 
reading the collocation “fossil fuel reserves"] [syntactic parsing]. 


The three remaining strategies on the lower processing level, as presented in Table 
7.2, were the additional categories added to the a priori coding scheme after the 
inductive coding cycle. These categories — emotional resonance, mother tongue 
interference, and vocabulary and grammar learning — mostly concerned fixations 
on single words. However, in some instances these strategies also included look- 
backs at the preceding context in the sentence, similarly to the syntactic parsing 
and establishing propositional meaning strategies. Emotional resonance was 
reported by participants in the instances when the information they were reading 
in the text surprised them or resolved a previously held misconception about the 
fact described in the text. In some cases, participants also reflected on fixating on a 
word because they had strong emotional associations with it: 


Participant 6: I think I focused on the temperature in the universe -273C, 
because I thought: oh, my God! How many degrees is that! 


Mother tongue interference was closely connected with the lexical access process- 
ing described earlier. Among the reasons participants gave for fixating on certain 
words was the need to translate these words into their mother tongue to better 
understand their meaning. As part of the final category within lower-level process- 
ing, vocabulary and grammar learning, participants associated the long fixations 
they had during reading with their attempts to learn the use of certain language 
structures in the text: 


Participant 7: I think I paid attention to such word combinations as "have 
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been built" and *in the long term", "they will" because I want to understand 
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better the grammar topic of tenses and time indications in English. It was 
interesting for me. 


While lower-level processing included the strategies participants took to under- 
stand the text on a sentence level, the three categories within the higher-level 
processing — inferencing, building a mental model, and creating text level represen- 
tation — were concerned with participants integrating sentences in the text together 
into a cohesive whole. 

Participants used inferencing strategy when they tried to activate their back- 
ground knowledge of the topic of the text to understand better what was being 
read, or to interpret the meaning of a series of words in a paragraph using their 
everyday experience. Similar to the emotional resonance processing, inferencing 
mostly concerned factual information in the text: geographical places, numbers, 
historical events: 


Participant 2: When I read “Gulf War", at first, I confused it with the WWII 
but then I realized it is a different event. 


Building a mental model was used by participants to either confirm their under- 
standing of how ideas in the text were developed, or to resolve any conflicting 
understanding they had when reading the different parts of the text. In contrast, the 
strategy of creating text level representation was mostly used when participants tried 
to apprehend what the text was going to be about when reading the title and open- 
ing sentences, or to rehearse the key points in the text to remember them better: 


Participant 5: When I finished reading the text, I looked back at any numbers, 
places, factual info that the text contained to make sure I remember them, as 
well as the last two sentences in the text. Usually these are the key points to 
take away from the text [creating text level representation]. 


A recurring observation that was made during the qualitative data analysis con- 
cerned the plasticity of cognitive processing. In cases when the use of one cognitive 
processing strategy did not facilitate text comprehension, participants reported 
having turned to another processing strategy to compensate for this failure. To 
exemplify, if lexical access was unsuccessful, participants would turn to syntactic 
parsing, establishing propositional meaning or building a mental model to make a 
better use of the context of the text: 


Participant 8: I don't know the word "overwhelmingly" and thats why I 
couldn't understand the preceding sentence and went back to re-reading the 
previous one [establishing propositional meaning]. 


7.4.2 Cognitive processing in authentic versus simplified OERs 


The analysis of the eye-tracking replays showed that there were more fixations and 
look-backs to discuss in each participants gaze plot video that corresponded to 
their reading of the authentic OER. Thus, in order to proceed to the analysis of 


98 Irina Rets et al. 


stimulated recall interviews and identify the frequency of use of each cognitive 
processing strategy, the number of each processing strategy from the content anal- 
ysis was divided by the total number of processing strategies. By calculating this 
relative measure, it was possible to control for this difference in the amount of 
cognitive processes verbalised by participants during their reading of authentic and 
simplified texts. 

The results of the stimulated recall data analysis are presented in Table 7.3, which 
gives an indication of the amount of use of the different cognitive processes in the 
total number of readings of authentic OERs (n = 9 readings / stimulated recall 
interview sessions) and simplified OERs (n = 9). No comparative statistics were 
run because of the relatively small sample size in this research. 

Overall, the most used processes for both authentic and simplified texts, as 
judged by the relative frequencies of categories’ occurrence in participants’ verbali- 
sations, were lower-level processes — lexical access, establishing propositional mean- 
ing and syntactic parsing. When exploring the differences in the frequency of use 
of different cognitive processing strategies for authentic vs. simplified OERs, three 
main differences became apparent. The key difference was in the amount of use of 
lower- vs. higher-level processing. Participants seemed to rely substantially less on 
lower-level processing when reading simplified OERs, as compared to their read- 
ing of the authentic texts. As has been outlined in the previous section, lexical 
access, syntactic parsing, and establishing propositional meaning were mainly 
employed when participants tried to resolve confusion in understanding the mean- 
ing of single words or clauses they encountered in the text. Using fewer strategies 
within these three categories for simplified OERs might indicate that participants 
experienced fewer comprehension difficulties, and their reading of the simplified 


Table 7.3 Stimulated recall data analysis results: authentic vs. simplified OERs 


Level of processing Category Authentic OER Simplified OER 
n=9 n= 
Lower level word recognition 7 3 
lexical access 66 43 
syntactic parsing 40 28 
establishing 45 36 
propositional 
meaning 
emotional resonance 7 N 
mother tongue 4 5 
interference 
vocabulary and 6 0 
grammar learning 
Higher level inferencing 5 12 
building a mental 15 25 
model 
creating text level 6! 10 


representation 
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OERs was smoother. In contrast, increased use of higher-level processing, 
particularly inferencing and building a mental model, when reading the simplified 
OERs indicates that participants had more capacity for connecting the text to their 
personal experiences and focus on the main themes of the text. Among the higher- 
level processing strategies, creating text level representation was used slightly less 
when reading the simplified OERs. A potential explanation of this finding could 
be that participants did not have to take an additional effort, reread and rehearse 
the text as they might have had more capacity to understand the text well during 
their initial reading. 

The other two differences in the use of processing strategies for authentic vs. 
simplified OERs concerned the frequency of use of the strategies that were added 
to the coding scheme after the inductive coding cycle. On the one hand," vocabulary 
and grammar learning" was mentioned only with the authentic OERs. On the 
other hand, as can also be seen from Table 7.3, the use of the strategy “emotional 
resonance" was slightly higher for the simplified OERs, as compared to the 
authentic texts. The latter finding suggests that, similarly to the case with higher- 
level processing, participants might have had more working memory capacity 
available to ponder over the simplified texts and to resolve a previously held 
misconception about a fact described in the text, or to draw stronger emotional 
associations with it. The frequency of appearance of *mother tongue interference" 
strategy was largely similar between the reading of authentic and simplified OER, 
which suggests that at times participants turned to the resources of their mother 
tongue to understand the texts, irrespective of the complexity of these texts. 


7.5 Discussion 


Open educational resources (OERs) are learning, teaching and research materials in 
any format and medium that are freely available in the public domain. Although 
pioneered with the intent to widen access to education globally, very few studies 
explored solutions on how to improve their accessibility to non-native English 
speakers (NINES). Chapter 7 aimed to obtain emerging evidence on the effect of 
OER text simplification on text processing of NNES at lower levels of proficiency, 
using qualitative evidence from eye-tracking. To that end, this research focused on 
comparing the frequency of use of different cognitive processing strategies at lower- 
and higher-levels of processing, as verbalised by participants in the eye-tracking 
stimulated recall sessions after they had read an authentic and a simplified OER. 
Chapter 7 showed that participants engaged in a wide range of cognitive pro- 
cessing when reading both authentic and simplified texts. This finding is partly in 
line with the earlier test validation studies that used eye-tracking stimulated recalls 
(e.g., Brunfaut & McCray, 2015) and showed that the entire spectrum of processes 
specified in the central core of the Khalifa and Weir (2009) model were elicited 
by the test questions during reading.Yet, Brunfaut and McCray (2015) also found 
that the frequency of use of lower- and higher-level strategies was largely similar 
across the sample. In contrast to this research, the research at the centre of this 
chapter identified proportionally lower reported usage of higher-level processing 
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(inferencing, building a mental model and creating text level representation) when 
reading both authentic and simplified OERs. 

Overall, the most used processes in this research were lexical access, syntactic 
parsing and establishing propositional meaning, as evidenced in the stimulated 
recall data. This finding might be due to the fact that participants knew there 
would be no comprehension assessment after reading. Reading the text for an 
immediate comprehension test is likely to have elicited a wider use of different 
cognitive processing strategies when reading authentic and simplified texts. This 
finding might also be due to the proficiency level of participants in this research. 
As has been shown in the study of Brunfaut and McCray (2015), participants at 
lower levels of language proficiency used lower-level processing strategies more 
frequently than participants at higher levels of proficiency. This could be the case 
in this research, where all recruited participants were from an intermediate (B1) 
English language course. 

In the comparison of the frequency of use of different cognitive processing strat- 
egies when reading authentic vs. simplified texts one key difference was observed 
in the amount of use of lower- vs. higher-level processing. Although lower-level 
processing was still dominant, participants seemed to rely less on the use of lower- 
level processing in the simplified OER. To exemplify, the use of “lexical access” 
strategy implied that participants made an effort to understand the meaning of a 
word in the sentence (Khalifa & Weir, 2009). The less frequent use of this strategy 
during the reading of the simplified OERs suggests that participants had fewer 
points of confusion or doubt about the meaning of a word, as compared to their 
reading of the authentic texts. This tendency was also observed when exploring the 
replays in the eye-tracking software which showed that there were fewer areas in 
the simplified texts where participants had to stop and make long fixations. 

Chapter 7 also provided some empirical evidence that text simplification facili- 
tated higher-level text processing. The categories that concerned higher-level pro- 
cessing occurred more frequently in participants! verbalisations for the simplified 
OERs, as compared to their reflections on authentic OER reading. It can be 
assumed that participants had fewer instances where they had to use lexical access 
and other lower-level processing strategies to understand the text on a sentence 
level. Besides the use of lower- vs. higher-level processing, another difference in 
processing of authentic vs. simplified OER concerned the frequency of use of the 
"emotional resonance" strategy. Since in this research "emotional resonance" 
referred to the instances where participants talked about feeling surprised, as well 
as about their emotional associations or ability to learn a new fact from the text, 
this strategy can also represent situational interest. Situational interest is defined as 
a relatively short-lived psychological state of focused attention, curiosity, and posi- 
tive affect (Soemer & Schiefele, 2019). When defined through the lens of situa- 
tional interest, the evidence from this chapter concerning the increase in emotional 
resonance when reading the simplified OERs is in line with the study of Soemer 
and Schiefele (2019). The authors showed that more difficult texts were perceived 
by the readers to be less interesting, and less interest, in turn, was associated with 
reduced focus of the readers towards the text. The finding of this research on 
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increased emotional resonance to the simplified text suggests that text simplifica- 
tion provides opportunities for the creation of stronger bonds between linguistic 
and emotional content, which is an aid for foreign language reading. Drawing from 
the aforementioned piece of evidence in the literature (Soemer & Schiefele, 2019), 
higher emotional resonance suggests a positive effect of text simplification on text 
processing among NNES. 

The final difference in processing of authentic vs. simplified OERs was the use 
of the strategy "vocabulary and grammar learning". This strategy was mentioned 
only with the authentic texts. The reason for that might be a higher lexical diver- 
sity of the authentic OERs, which might have given participants more instances of 
exposure to various lexis and grammar structures. Thus, this finding suggests that 
simplified texts may limit incidental vocabulary learning. However, since learning 
with OERs is primarily concerned with subject content comprehension, rather 
than with language acquisition, incidental vocabulary learning might not have 
immediate relevance in this context. 


7.5.1 Implications for practice 


Chapter 7 provided emerging evidence in support of the use of text simplification 
to increase linguistic accessibility of OERs to NNES. The important practical 
implication from this research is to encourage OER material writers to check the 
text complexity level of their materials prior to publication and to linguistically 
simplify them, where possible. Simplification strategies such as splitting sentences, 
choosing words of a shorter length and higher frequency, using fewer nouns and 
more connectives between/within sentences have a beneficial effect on the text 
processing of NNES. Simplification stimulates greater focus and more interest 
towards the content of the text. As long as the linguistic accessibility of open edu- 
cation is being ignored, and OERs continue to draw on native speaker capital in 
language, the capacity of these resources to widen access to quality education will 
only remain that: a potential. 
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Chapter 8 


Culturally adaptive learning design 
A mixed-methods study of cross-cultural learning 
design preferences in MOOCs 


Saman Rizvi, Bart Rienties, Jekaterina Rogaten 
and René Kizilcec 


8.1 Introduction 


Over the past decade, open online learning environments have changed the edu- 
cational landscape all around the world. Increasingly, formal degrees are taking a 
hybrid form or being replaced by digital literacy products, such as Massive Open 
Online Courses (MOOCs) (Shah, 2019). MOOCs, as large-scale, freely accessible 
learning environment, are primarily recognized for their potential to facilitate 
universal learning access, as previously identified in Chapters 5 and 6 (Chua, 2022; 
Conde Gafaro, 2022). A learner can learn from these courses as long as they have 
access to appropriate resources such as a computer, laptop or mobile device and an 
adequate internet connection (Jansen & Schuwer, 2015). Still, emerging data sug- 
gest varied persistence and achievement gaps for learners from various regions 
(Reich & Ruipérez-Valiente, 2019). In contrast to the expectations of MOOC 
enthusiasts (Bozkurt & Aydin, 2018; Jansen & Schuwer, 2015), there is substantial 
inequality and disparity in the global digital learning landscape, with regional and 
cultural backgrounds influencing the way learners engage with MOOCs (Guo & 
Reinecke, 2014; Kizilcec & Halawa, 2015; Kizilcec et al., 2017; Ogan et al., 2015; 
Reich & Ruipérez-Valiente, 2019). 

The way MOOCs are designed — in short learning design (LD) — can substan- 
tially influence learners’ persistence in MOOCs. Typically, in MOOCs this entails 
various types of learning activities, offered in a predetermined order. Recent litera- 
ture suggests that a centralised LD containing prearranged, fixed number of activi- 
ties, may not work for all learners (Bearman, Lambert, & O'Donnell, 2021). 
Additionally, LD and other pedagogical factors (e.g., teaching methods and con- 
tent) may have a predictive and causal link with learners’ progression and whether 
(or not) they stay in the course (Xing, 2019; Guo, Kim, & Rubin, 2014). However, 
there is limited focus on how and to what extent the influence of LD varies with 
geo-cultural contexts. Previous work suggests that various geo-cultural groups 
have a distinct preference for particular learning activities, but the research is lim- 
ited on the ways to adapt and tailor LD accordingly (Joksimović et al., 2017). The 
overall results from this research helped us to understand the association between 
learning activity types and learners’ persistence in MOOCs. In Chapter 8 we will 
explore how such association varies between geo-cultural contexts. 
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8.1.1 Learning design and learning behaviour 


The critical role of pedagogical factors, such as LD, in learner persistence has been 
widely acknowledged in formal learning environments (Nguyen, Rienties, & 
Toetenel, 2017; Rienties & Toetenel, 2016; Rienties, Nguyen, Holmes, & Reedy, 
2017) as well as in MOOC learning environments (Rizvi, Rienties, Rogaten, & 
Kizilcec, 2020; Xing, 2019). In Chapter 8, we conceptualise LD as a course devel- 
opment process, i.e., a process of designing a series of learner-facing activities. The 
process produces a course as a sequence of learning activities of different types (e.g., 
reading material, instructional audios, videos, and discussions). The designed activi- 
ties can be reused when needed. To the best of our knowledge, MOOC learning 
designers tend not to modify or adjust the course LD once the course has been 
offered. Few studies have examined learners’ interaction with MOOC learning 
resources, specifically with various content types, for example, text-based resources 
(Ruzvi et al., 2020; Uchidiuno, Koedinger, Hammer, Yarzebinski, & Ogan, 2018), 
instructional videos (Davis, 2019; Guo et al., 2014), course assessments (Juhanak, 
Zounek, & Rohlikova, 2017; Li & Baker, 2018), and participation in discussion 
forums (Allon, Van Mieghem, & Zhang, 2016; Sunar, White, Abdullah, & Davis, 
2016; Yang, Sinha, Adamson, & Rosé, 2013), as also indicated in Chapter 6 (Chua, 
2022). Recent work suggests that a centralised LD containing a fixed number of 
sequenced learning resources may be convenient and even be beneficial for most 
learners, but this does not guarantee that it will be useful for all learners (Bearman 
et al., 2021; Margaryan, Bianco, & Littlejohn, 2015). 

In Chapter 8, we leveraged FutureLearn MOOCs as a primary source of data, 
where the basic course element in LD is called a step. This step represents a learning 
activity in a MOOC and could be of several types: Article, Discussion, Peer Review, 
Quiz, Text, Video/Audio, Exercise (Sharples, 2015). The content and structure are 
designed in accordance with the course needs and then activities are grouped 
together in a sequence. A title is used to describe the overall learning objective of 
that group. As illustrated in Figure 8.1, most FutureLearn MOOC designs primar- 
ily contain four types of activities; Articles, Videos, Discussions and Quizzes 
(Sharples, 2015). 

For the learning activity categorisation, we used the OULDI theoretical frame- 
work (Cross, Galley, Brasher, & Weller, 2012), which is further described in 
Chapter 14 (Nguyen, Rienties, & Whitelock, 2022). According to the OULDI 
framework, reading articles or watching videos are referred to as assimilative 
activities, i.e., learning activities to develop, process, and attain information in an 
online course. Next, discussions are categorised as communication-based activi- 
ties which allow learners to participate in course-related discussions. It is impor- 
tant to highlight here that although discussion-steps are considered an integral 
part of LD, FutureLearn MOOCs also offer a commenting space underneath 
every learning activity that comprise a social media-style feed. In the comment- 
ing space, a learner can start, like or comment on a discussion or follow their peers 
and instructors (Sharples, 2015). The FutureLearn platform explicitly embeds dis- 
cussions in the LD of all courses and aims to make MOOCs a social learning space 


Week 1: Introduction & General 
Theory 


An overview of important background concepts and theory. This will provide 
the framework for you to locate the various algorithms and methods we 
look at in the remaining weeks. 


Drag and drop to re-order -+ Add an activity 


Introduction (3 steps) P d 
Data Science Workflow (2 steps) x 
Types of Learning (3 steps) x 
Linear Models, Non-Linear Models & Feature Transformations (3 steps) x 
Expected Loss, the Bias-Variance Decomposition & Overfitting (4 steps) 2€ 


Statistical Models, Loss Functions and Training as Optimization (3 steps) 
Regularization (5 steps) x 


Week One Wrap Up (2 steps) x 


Expected Loss, the Bias-Variance 
Decomposition & Overfitting 


We examine the idea of the expected loss of a statistical 
model, look at the bias-variance decomposition of 


expected loss and discuss the relation of this to 
overfitting. 


Z Edit this activity 


Drag and drop to re-order 


112 Expected Loss, the Bias-Variance Decompostion & Overfitting 
113  Bias-Variance 
- 
114 Do you understand: Bias, Variance & Overfitting lau) 
115 Discussion on Expected Loss, Bias-Variance and Overfitting EB 


Figure 8.1 An overview of one set of activities in a FutureLearn MOOC. 
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(Manathunga, Hernandez-Leo, & Sharples, 2017). Finally, assessment activities such 
as Quizzes were taken into consideration, as the role of assessments is widely rec- 
ognised as critical to the learning and engagement. We have been specifically 
focussing on these four main learning activity types. 

This research was driven by the motivation that limited research has explored 
how different proportions of the various learning activity types (1.e., reading mate- 
rial, videos, quizzes and discussion-based activities) can be potentially linked with 
MOOC learners’ persistence, and that most of the existing research fails to address 
learners’ perceptions about these activity types. Ideally, there are several pedagogical 
factors that can be made flexible and modifiable either midway or between course 
runs. These factors include learning activity types, sequence of those activities, and 
content difficulty level. Therefore, it is imperative to understand how learners’ per- 
sistence is linked with these learning activities in a course and then unpack the 
learners’ perspectives about these activities. 


8.1.2 Geo-Cultural background and learning behaviour 


In the context of participation in MOOCs, several researchers have found vast 
regional and cultural differences in behavioural engagement and persistence (Reich 
& Ruipérez-Valiente, 2019; Kizilcec & Halawa, 2015). For example, studies 
reported differences in course assessments (Liu et al., 2016; Kizilcec & Halawa, 
2015), video watching behaviour (Liu et al., 2016; Uchidiuno et al., 2018) and 
social interactions within a course (Liu et al., 2016; Ogan et al., 2015). Between 
various geo-cultures, distinct learning patterns have also been noticed in reading 
versus video-watching behaviour (Uchidiuno et al., 2018; Liu et al., 2016; Reinecke 
& Bernstein, 2011). One way to approach this issue could be by designing an open, 
online course that adapts itself to the dynamic cross-cultural needs. 

In line with previous research (Mensah & Chen, 2013; House, Hanges, Javidan, 
Dorfman, & Gupta, 2004), we used the GLOBE geo-cultural framework for learn- 
ers’ categorisation. This framework distinguishes global regions and their cultural 
constructs by categorising them into ten culturally similar clusters: Sub-Saharan 
Africa (AF), Anglo-Saxon (AS), Confucian Asia (CA), Eastern Europe (EE), 
Germanic Europe (GE), Latin America (LA), Latin Europe (LE), Middle East (ME), 
Nordic Europe (NE), and Southern Asia (SA). By categorising learners using these 
ten clusters, first we examined the association between the number of learning 
activities and learners’ persistence in the MOOC. Second, we explored certain 
activity types that were an enabler for one geo-cultural group while limiting for 
another. Lastly, in a follow-up study, we explored the learners’ experiences and 
their views on various activity types. 


8.2 Case studies 


In the first study (Rizvi et al., 20213), we used a quantitative approach to inspect 
trace data for learners enrolled in ten large FutureLearn MOOCs (n = 49,582). 
The sample was diverse and heterogenous, with learners from all ten geo-cultural 
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regions. The largest subgroup belonged to the Anglo-Saxon countries, closely fol- 
lowed by South Asian, and African learners. The smallest subgroup originated from 
Nordic Europe. We examined whether (or not) differences regarding the number 
of assimilative activities (articles and videos), communication activities (discus- 
sions), and assessment activities (quizzes) within a MOOC could be used to predict 
learners' persistence. Next, we compared the predictive associations between the 
ten geo-cultural groups. 

It is noteworthy that most quantitative methods remain biased in favour of the 
largest subgroup that exists in the data. Therefore, empirical studies often reflect the 
results fitting to the needs of the largest subgroup (Anglo-Saxon participants in our 
case), which may (not) be appropriate for other subgroups. To overcome this meth- 
odological issue, we used interaction terms in our analysis which takes into consid- 
eration the geo-cultural subgrouping as well as predictors, such as the number of the 
various learning activity types across the ten MOOCs. Advanced statistical methods 
associated with survival analysis were used to predict the outcome variable persistence. 
Persistence represents the learners’ progression in the respective course using the 
percentage of course activities accessed by a learner before they dropped out. 

The follow-up qualitative study utilised semi-structured interviews to collect 
information on learners’ perceptions about the various types of learning activities 
in FutureLearn MOOCS. To understand these varied perceptions, several in-depth 
interview questions were used, for example, Which learning activity type (article, video, 
quiz and discussion) did you enjoy most/least? While we had a sample of 22 partici- 
pants from seven geo-cultural groups, we were unable to recruit participants from 
Confucian Asia (CA), Latin Europe (LE) and Nordic Europe (INE). The study 
employed well-established and widely used method of thematic analysis (Braun & 
Clarke, 2006) to understand participants! perspectives and experiences. It is impor- 
tant to note here that we have only shared some of the relevant quotes, describing 
participants’ experiences with the respective activity type, and not the entire out- 
come of the thematic analysis. The detailed results can be accessed via other publi- 
cations that relate to this study (Rizvi et al., 2021b). 


8.3 Selected findings 


In order to explore if changing the number of learning activity types is associated with 
learners’ greater persistence in MOOCs, we used a number of various learning activ- 
ity types to predict learners’ persistence in the respective FutureLearn course (Rizvi et 
al., 20212). We quantified the predictive link and found distinct links for each of the 
activity types. The findings suggest that irrespective of the geo-cultural background, a 
large number of learning activities in a course design was not liked by most learners. 


8.3.1 Assimilative activities: articles 


The LD of most MOOCs examined in this study tended to include one or more 
reading activities that either contain reading material or links to other reading 
resources, or both. We found that increasing the number of reading activities was 
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associated with an increased risk of dropout. For the dataset we used, the analysis 
suggested an increased dropout risk of 14% for every 20 short reading steps added 
in a course, if the course already had around 52 such reading steps. The interaction 
analysis suggested that this dropout risk was most severe (and statistically signifi- 
cant) for learners from Latin American region (48%), followed by learners from 
Anglo-Saxon (28%) and African (7%) regions. During the follow-up interviews, 
this is how some participants shared their experience of reading activities. 


I didn’t perceive articles as reading content. And sometimes you can come 
across quite dense written word or content there. It has to be engaging, 
because it can be almost like reading a newspaper article. Yeah, quite long in 
length? I tend to find them to be a little bit, for-information-purpose-only 
type of thing, and not necessarily engaging. 

(Participant13, Male, AS) 


The quantitative results suggested that non-native English speakers, particularly 
from South Asian and Middle Eastern regions, were relatively less affected by the 
large number of articles in some courses. However, they deemed articles as boring 
or even unnecessary at times. Language barriers were mentioned frequently by non- 
native English speakers, as something negatively influencing their engagement 
with reading activities, as was also found in Chapter 7 (Rets, Stickler, Coughlan, & 
Astruc, 2022). 


When you're studying (from an article) in [participant’s native language], you can 
pick it quite in a limited time. But when it’s in English, it takes you time to 
pick up those points and absorb that information. 

(P6, Male, SA) 


We found this result to be aligned with the previous literature that suggests that 
learners from non-English-speaking background tended to spend more time on 
assimilative (reading, watching) type of learning activities in online courses 
(Nguyen, Rienties, & Richardson, 2020). 


8.3.2 Assimilative activities: videos 


MOOC learning environments are generally recognised for their video lecture- 
based LDs. During the overall data analysis, we found a small yet significant link 
between the number of videos and persistence. Taking into consideration the pres- 
ence of our ten geo-cultural groups, the link was not only quantifiable but also a 
large significance was noticed for several groups, particularly for South Asian learn- 
ers. In other words, every increase of 9 videos in a course reduced the dropout risk 
for South Asian learners by 6% (given that the course already contained around 22 
short instructional videos). In contrast, a small negative association was found 
between the number of videos and persistence for Anglo-Saxon learners, but fur- 
ther analysis found the risk to be not statistically significant. The most significant 
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association was found for Middle Eastern learners (996 increase in dropout risk). 
Furthermore, the interviews indicated that participants from Anglo-Saxon and 
Germanic Europe regions consistently reported their dislike of instructional videos 
in FutureLearn MOOCs, finding most of the videos to be far from engaging, per- 
haps foo slow for their taste. 


I am less likely to watch the videos as I am more likely to read the [video] 
transcripts...I think if I am watching a video, I am more likely to lose focus 
and to kind of, for my thoughts to like, kind of drift somewhere else. 

(P3, Female, AS) 


Participant 16 echoed this opinion, 


I mean they [videos] are slow because they are always speaking very clearly, 
and slowly to make sure that you understand. Well, I’ve now lost my focus and 
I'm already at some other planet. It’s just far too slow for me. It doesn't work 
and if you speed it up, gets on mentally, really weird! So, it doesn’t work. 
Speeding it up doesn't work. So, that's why I dislike videos that are just far too 
slow for me. 

(P16, Female, GE) 


Learners from Anglo-Saxon regions tended to dislike instructional videos. In con- 
trast, South Asian learners reported a strong preference for learning from videos. 


The most favourite part I enjoyed is watching videos, the HD videos, which 
was just virtually...I was thinking as teacher is just teaching me sitting in front 
of me or I'm sitting in a virtual classroom and learning. 

(P14, Male, SA) 


8.3.3 Communication activities: discussions (instructor-led 
I| user-led) 


Almost all mainstream MOOC providing platforms now feature a social learning 
space in form of either a separate discussion forum, or a discussion space located 
directly underneath every learning activity (FutureLearn design approach) or both. 
Overall data analysis suggested a rather small, negative association between the 
number of discussions and persistence in the course (a 3% decrease in dropout risk 
with six more discussion-based steps added in a course already containing 14 dis- 
cussions). A subgroup analysis suggested that the impact was again dissimilar across 
the various geo-cultural subgroups. For example, a negative association between 
early dropout risk and number of discussions was found for learners from Anglo- 
Saxon, Confucian Asia, Nordic Europe, Germanic Europe, Latin Europe and Latin 
America, with learners from these geo-cultural groups engaging less with the 
courses containing fewer discussions. In contrast, African and South Asian learners’ 
did not favour a large number of discussion steps in a MOOC LD (ze., early 
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dropout risk increased by 9% and 23%, respectively). Follow-up interviews sug- 
gested that learners from geo-cultural regions who were interested in discussions 
were still more inclined towards user-led discussions, and not towards instructor- 
led discussion. However, we found several differences in opinion within and across 
various geo-cultural groups, and our overall analysis remained inconclusive (Rizvi 
et al., 2021b). Still the respondents provided useful insights into their views on 
communication and social interaction in MOOCs. 


I guess online, you might have thousands of people, making a point in front of 
thousands of people. It is completely different because there’s very little chance 
that many of them will be listening or paying attention. So, I guess if it’s a 
really large group, I feel more comfortable with that. 

(P17, Female, AS) 


In contrast, the large number of discussants participating asynchronously was an 
issue for others. 


When you don't have time to engage every day, by the time you would log on 
the discussion, they would already be 20 or 30 posts. 
(P20, Male, AF) 


Furthermore, a lack of privacy or agency over one’s comments was another con- 
cern raised by most participants, but primarily by female participants from Middle 
Eastern or Eastern European regions. Other participants identified a need for fre- 
quent interaction from the instructors' side to improve engagement, along with an 
inclusion of social-media style features in MOOC commenting spaces. 


There was another thing I think should be included more often, and it is for 
example, participating in the discussion forums. I like using for example, the 
symbol at (@), like tagging people. So, they know that I’m mentioning them 
in my in my comment. But it’s not that easy to come up with. Sometimes it 
works, sometimes it doesn’t. But I think that is a good way to engage other 
learners in the conversations we are having in the discussion forums. 

(P1, Female, LA) 


8.3.4 Assessment activities: quizzes 


Assessment activities are considered an essential part of the learning process, even 
in flexible, self-paced learning environments like MOOCs. However, in our 
research an increased number of assessment activities (i.e., quizzes) was found to 
have a negative association with learners’ persistence in the respective course. With 
the sole exception of South Asian learners, the pattern was the same across all geo- 
cultural subgroups. We found, for example, that adding seven more quizzes in a 
course that already had around seven quizzes, tended to increase the average drop- 
out risk by 15%. As discussed before, this pattern did not mirror the view of the 
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second largest subgroup of South Asian learners, where the association was positive, 
slightly favouring more quizzes in MOOC LD.The large elevated risks we noticed 
were for learners from Middle Eastern, African and Anglo-Saxon countries (7%, 
9%, and 21%, respectively). The follow-up analysis to explore learners’ perceptions 
about quiz-based assessment activities in MOOCs revealed various dimensions. 


I do like quizzes. I think quizzes can give you a real sense of you know... One, 
they are fun, and two, it's good to sort of check. So, I think the quizzes are 
important because in some ways I know it sounds very old school, but like 
quizzes and tests and so on, are probably part of my own experience of 
education. 

(P12, Female, AS) 


Since in MOOCs most learners tended to prefer a pick-and-choose learning 
behaviour, they remained hesitant on being quizzed on the content they might 
have missed. But often learners did not consider frequent assessments to be a useful 
part ofthe MOOC LD. 


I'm not there to be tested on, I would like to, you know, to discover new 
things. But I don't really like to feel that I am tested upon. 
(P21, Female, EE) 


8.4 Discussion 


In Chapter 8, first we aimed to explore the predictive link between the number of 
different types of learning activities in an LD and learners’ persistence in 10 
FutureLearn MOOCs followed by 49,582 learners. While doing so, we also exam- 
ined the extent to which the link between LD and persistence, differed between 
geo-cultural contexts. Second, we scrutinised learners’ perceptions and their self- 
reported experiences with various types of learning activities using 22 in-depth 
interviews. As a whole, most learners preferred to have fewer rather than more 
activities in the LD of an MOOC.A notable exception was learners from South 
Asian countries, who chose to engage longer with MOOCs that contained a large 
number of small bite-sized learning activities. In contrast with previous work that 
pointed towards the critical role of discussions in MOOC learning (Manathunga 
et al., 2017; Allon et al., 2016), Chapter 8 found that LDs which provided many 
opportunities to interact with peers by instructing learners to discuss certain course 
topics actually averted active participation of learners from non-English-speaking 
geo-cultural regions, such as Sub-Saharan Africa and South Asia. We found that 
such an approach only slightly supported learners from Anglo-Saxon and European 
regions (GE and LE). 

In addition, we found learners overall persisted more in the courses containing 
a greater number of assimilative learning activities (i.e., articles and videos), specifi- 
cally those with instructional videos. As all the content in the ten MOOCSs we 
analysed was offered in English, we expected greater engagement from native 
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English-speaking learners. However, most learners were found to disapprove of 
large amounts of reading materials. Increasing the number of articles increased the 
early dropout risk for all learners, even for those residing in the Anglo-Saxon 
regions and neighbouring regions (such as Germanic and Latin Europe). 

Since most early MOOCs were offered in a video-lecture format, instructional 
videos have long been assumed to be a central feature in a MOOC LD. Our analy- 
sis of ten MOOC LDs with a varied number of videos revealed a minimal link of 
increasing the number of videos with learners’ persistence. The only significant 
link we found was that only South Asian learners engagement lasted longer by 
increasing the number of videos. Our qualitative analysis revealed that learners had 
contrasting opinions about course videos. For example, in line with Uchidiuno 
et al. (2018) who found non-native English-speaking learners to engage least with 
the videos that contain narration with no other visual support, Middle Eastern 
learners reported a desire to learn from either vibrant videos or from something 
"richer" than the videos (i.e., detailed informative articles) and learners from 
Anglo-Saxon regions and from Germanic Europe deemed clear, slow-paced vid- 
eos to be disengaging. 

Participants from all around the world consistently raised a need for more inter- 
active videos or videos with embedded quizzes. Concerning the assessment activi- 
ties in MOOCs, we found learners to persist more in the courses that offered 
quizzes in moderation. While learners from English-speaking and European 
regions liked to be quizzed in moderation, a slightly negative yet statistically strong 
association was found between the number of quizzes and persistence of learners 
from South Asian and Middle Eastern groups. 


8.5 Limitations and moving forward 


There are several limitations with the approach we used in Chapter 8. For example, 
all MOOCs used during the quantitative analysis were offered via the same 
MOOC platform (FutureLearn) and were designed by the same LD team at the 
Open University, UK. Moving forward, a better approach would require experi- 
mental manipulation of LDs, possibly during and between the course runs. A cross- 
platform analysis might also yield different results. Also, since all MOOCs under 
analysis were offered in English language, a comparison of monolingual versus 
multilingual MOOCs could reveal different patterns of engagement in dissimilar 
geo-cultural contexts. 

As for the qualitative analysis phase, all semi-structured interviews were con- 
ducted in English, while a large number of participants (16 out of 22, or 73%) were 
non-native English speakers. The odds remained high that those participants might 
have struggled to verbalise their thoughts when asked about their experiences with 
the LD. Finally, the only demographic factor that was taken into consideration in 
Chapter 8 was learner’s location at the level of the geo-cultural region. There could 
be several other individual and demographic factors potentially influencing learners? 
persistence and their overall experience with the course LDs. Such factors include 
age, gender, socio-economic status, and employment level. We acknowledge that 
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these factors can be part of learners’ broader cultural experiences, but these factors 
are beyond the scope of current research. Along with the cultural dimensions, ana- 
lysing the socio-economic and individual factors may yield interesting insights as 
few participants themselves pointed out. 


Is there a standard African learner? Do you prefer them to be English language 
speaker, second speaker or third speaker? Do you prefer them to be male and 
unemployed or female and pregnant? 

(p20, Male, AF) 


8.5.1 Implications for practice 


Our findings suggest that the link between persistence and changes in LD (chang- 
ing number of various types of activities) varies with the geo-cultural context. 
Perhaps there is no ideal combination of learning activities that facilitate learners 
from all around the world and Chapter 8 provides some explanations as to why 
there is no single, universal LD for MOOC that can work for all learners. We found 
that a fixed, predetermined LD can hardly be inclusive, and our qualitative results 
echoed the quantitative findings. Until we reach the (difficult yet attainable) mile- 
stone of a flexible, culturally adaptive MOOC LD, we recommend taking a bal- 
anced approach by combining different types of learning activities, not just 
video-based, or reading MOOCs. Despite the fact that development of culturally 
adaptive MOOCs may not always be cost-effective, cultural adaptation in designs 
of open online learning environments is still strongly recommended, not automati- 
cally perhaps, but if chosen by the learners. 
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Chapter 9 
Accessibility in MOOCs 


The stakeholders' perspectives 


Francisco Iniesto, Patrick McAndrew, Shailey Minocha 
and Tim Coughlan 


9.1 Introduction 


The reported work on how accessibility is embedded in the design of Massive 
Open Online Courses (MOOCs) seems to be limited (Sanchez-Gordon & Luján- 
Mora, 2017). At the same time, the need to incorporate greater access has been 
highlighted. Two significant events have contributed to the call for attention to 
the accessibility of online learning including MOOCs. The first event took place 
in 2015 when edX, a MOOC provider, had to reach an agreement to include 
accessible content on its platform. edX decided to change its policies and include 
accessibility in its software development cycle. The following excerpt from the 
settlement includes arguments that continue to be important (US Department of 
Justice, 2015, p. 1): 


MOOCs have the potential to increase access to high-quality education for 
people facing income, distance, and other barriers, but only if they are truly 
open to everyone. This landmark agreement is far-reaching in ensuring that 
individuals with disabilities will have an equal opportunity to independently 
and conveniently access quality higher education online. 


The second event in 2016 led to a different approach being taken by the 
University of California at Berkeley when facing a similar situation (Jaschik, 2016). 
The University of California decided to remove more than 20,000 audio and 
video files from its online open-access platforms, requiring learners to sign in with 
University of California credentials to view or listen to them instead of investing 
in making the content accessible. In both events, legislation was a driver for acces- 
sibility. Technologies used in MOOC platforms are not necessarily accessible 
(Sanchez-Gordon & Luján-Mora, 2017), which may then block access to learning 
for a significant number of learners. Furthermore, the use of videos within 
MOOC;, peer-to-peer assignments that involve learners evaluating each other’s 
contributions, quizzes, and tests, or, in general, activities that increase the need for 
collaboration in online learning all can create additional challenges for accessibility 
(Rodrigo, Iniesto, & García-Serrano., 2020), see also Chapters 5 and 6 (Chua, 2022; 
Conde Gafaro, 2022). 
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MOOCs can be beneficial when compared to other online learning opportuni- 
ties because of their characteristics of openness within a structured learning frame- 
work and low cost of learning (Iniesto, 2020). The scope of individual planning 
regarding learners’ self-organisation of time, the use of their preferred devices/ 
equipment, opportunities for social learning, and the chance to gain new knowl- 
edge are some additional advantages (Conde Gafaro, 2022). The importance of 
accessibility to online educational resources is widely acknowledged (Acosta & 
Lujan-Mora, 2016), but there is limited discussion about the accessible design of 
online learning courses including MOOCs. Providing accessible MOOCS could 
offer the flexibility of learning and benefits to all learners. Indeed, the Porto 
Declaration on MOOCs (EADTU, 2014, p. 2) highlights the aspect of providing 
opportunities to all: 


MOOCs must not be seen as the outcome or exemplar of online education. 
Rather they need to be understood in a wider context as there is a long his- 
tory of research on open and online education and a variety of approaches and 
tools to provide quality learning opportunities to all. 


A published report on inclusive teaching and learning in higher education 
(Department of Education, 2017) encourages higher education providers to care 
and offer support, and to develop an optimal environment for learners with acces- 
sibility needs. The lifelong learning paradigm integrates education, work, and per- 
sonal life in a continuous process and allows learners to be able to access knowledge 
and develop it both personally and through work (Butcher & Rose-Adams, 2015). 
In this respect, if accessible, MOOCS have the characteristics to provide an appro- 
priate mode of study for learners with accessibility needs. Chapter 9 summarises 
the doctoral research. programme which has yielded an understanding of how 
MOOC providers cater for learners with accessibility needs, the motivations of 
those learners when taking part in MOOCs, and how MOOCs should be designed 
to be accessible (Iniesto, 2020). 


9.2 MOOC Stakeholders and accessibility 


When reviewing the literature, there is a lack of research about the efforts that 
MOOC providers are making towards MOOCs accessibility, the number of learn- 
ers with accessibility needs and their interest in MOOCs, and the current state of 
MOCC accessibility (Iniesto, 2020). First, there is limited research on course team 
(authors) experiences of MOOCs, and how instructors/tutors are trained and 
interact with MOOC learners (Papathoma et al., 2020). In particular, little is 
known about how MOOC providers develop their platforms or courses by taking 
into account learners with accessibility needs. For example, Smith, Dowse, Soldatic, 
and Kent (2017) provided an overview of the process of developing a MOOC that 
included accessibility from the experiences of educators involved, whereby the 
authors aimed to explore what they named “disability pedagogy” in MOOCs. 
Smith et al. (2017) reflected that much of the work on MOOC development and 
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design was very ad hoc, showing how difficult it is to get development teams work- 
ing together. 

Second, in terms of the learners’ perspective, it is often difficult to know the 
actual number of learners taking part in MOOCs (Guo & Reinecke, 2014). 
However, the definition of success needs to relate to the learner and finishing the 
MOOC is not necessary the goal for all learners (Liyanagunawardena, Parslow, & 
Williams, 2017). The motivations of online learners are diverse, and specifically for 
the MOOCs, as also indicated in Chapter 8 (Rizvi et al., 2022). Some learners at 
university level showed particular interest in "having a full-time job" (Ilgaz & 
Gulbahar, 2017) while using online environments for social interaction or leisure 
(Serdyukov & Serdyukova, 2015). Particular research on learners with accessibility 
needs participating in MOOCs has been investigated by Liu, Kang, and McKelroy 
(2015) who highlighted the importance of usable MOOC designs because difficult 
navigation and unintuitive interfaces can have a negative effect on the learning 
experience and perceptions of the course. Similarly, Park, Jung, and Reeves (2015) 
reinforced the idea that MOOCs can be a challenging experience and should be 
as flexible as possible to meet the needs of diverse learners. 

Finally, the trend in many studies related to MOOC accessibility is towards tech- 
nical reports where accessibility is evaluated using human-computer interaction 
(HCI) techniques and the Web Content Accessibility Guidelines (WCAG, 2018), a 
de facto standard of web accessibility. In these studies, expert evaluation dominates, 
where one or more accessibility experts applied certain heuristics using automatic 
tools (Akgul, 2018). Other authors complement heuristic evaluations with user- 
participation in the assessment process, such as users who were partially sighted, or 
have other visual impairments (Królak, Chen, Sanderson, & Kessel, 2017), or the 
elderly (Bong & Chen, 2016). Most published studies report evaluating a single 
MOOC and corresponding platform and often involve vision impaired learners as 
participants. For a better understanding of the accessibility barriers in MOOCs, 
studies should be conducted that cover a combination of different accessibility 
evaluation methods and a broader sample of end-users with diverse accessibility 
needs and not just visual impairment. 

As Seale (2014) argued, there is a need to understand the multiple viewpoints of 
stakeholders in accessibility practice. Research with MOOC providers is needed to 
capture their practices and constraints of integrating accessibility. Furthermore, the 
motivations and barriers of the learners who have accessibility requirements need 
to be investigated. For example, Rodrigo and Iniesto (2015) argued the need to 
provide a holistic vision for creating accessible MOOCs. Therefore, several HCI 
accessibility evaluation methods are needed to evaluate MOOCs, to provide indi- 
cators of the accessibility barriers and to develop processes to be addressed. 
Therefore, the following research questions are addressed in Chapter 9: 


1 How do MOOC providers cater for learners with accessibility needs? 
What are the motivations of learners with accessibility needs when taking part 
in MOOCs? 

3 How can MOOCs be made accessible for learners with accessibility needs? 
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9.3 Researching accessibility in MOOCs 


The research has taken a study-based approach; each study having its own research 
design, and methods of data collection and data analysis. The results from the stud- 
ies are then related to each other and consolidated to address the research aims of 
the programme, and to draw out implications and directions for further work. The 
identification of appropriate methods for the research design was shaped by the 
ethical considerations of research that involved vulnerable groups involved with 
online learning (Farrow, 2016). 

A pragmatic methodology was selected for this research which included qualita- 
tive, quantitative and HCI methods. That pragmatic approach suggested is demon- 
strated in the literature on MOOC research. Liyanagunawardena, Adams, and 
Williams (2013) pointed out that in the initial stage of MOOCs, the period from 
2008 to 2012, the majority of research used multiple methods for data collection: 
primarily online surveys; and via interviews, focus groups and collecting platform 
analytics. Gasevic, Kovanovic, Joksimovic and Siemens (2014) proposed the use of 
mixed methods for research in MOOCs, by recognising the complexity of 
massiveness and openness of MOOCs. In a review that focused on 2013-2015 
literature, Veletsianos and Shepherdson (2016) indicated that researchers favoured 
quantitative approaches with survey data and secondary data collected via auto- 
mated methods, and qualitative methods informed few studies. Joksimović et al. 
(2018) in their systematic literature review pointed out the lack of generalisability 
of current results in MOOC research. As the literature review shows and Evans, 
Baker, and Dee (2016) suggest, research on MOOCs needs to focus on research 
approaches across different domains and multiple methods. 

In this research design, research methods that require collecting perspectives 
from stakeholders were included: qualitative studies with MOOC providers and 
learners with accessibility needs facilitated understanding of their views (RQ1 and 
R.Q2) and the way accessibility of MOOCS can be improved (R.Q3). Quantitative 
studies were used to understand the demographics and motivations of learners 
(R.Q2) and to draw out initial ideas on barriers (R.Q3).A systematic tool called the 
MOOC accessibility audit based on the heuristic evaluation method of expert 
usability evaluation in the HCI literature was developed in this project. The audit 
tool involved expert-based evaluation (conducted by usability/accessibility experts) 
to detect accessibility barriers (RQ3). The mapping between research questions 
and methods in the three studies are shown in Figure 9.1. 

In Study A, semi-structured interviews involved 26 MOOC providers and 
researchers, such as accessibility managers, course editors, inclusion designers, 
instructional designers, learning media developers and software developers (Iniesto, 
McAndrew, Minocha, and Coughlan, 2016). The aim was to explore the perspec- 
tives of platform and course providers on the importance of accessibility of the 
MOOC environment. The data from this study was useful to understand on how 
to approach the next components of the research programme. Interviewing indi- 
viduals involved in MOOC development (MOOC providers) helped to under- 
stand how they catered for learners with accessibility needs (R.Q1), and the 
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Figure 9.1 Research questions, studies and methods. 


approaches they used to design accessible MOOCs (RQ3). Thematic analysis 
(Lapadat, 2009) was chosen for (qualitative) analysis of the data from these explor- 
atory interviews. 

Study B employed pre- and post-online survey data from past 14 Open 
University’s MOOCs in FutureLearn (with 29,000 and 5,000 respondents) (Iniesto, 
McAndrew, Minocha, and Coughlan, 2017). The survey data was provided to the 
research team by FutureLearn who run pre- (at the start of the MOOC) and post- 
surveys (on completion of the MOOC) with their learners. The analysis of survey 
data provided preliminary insights related to research questions, RQ2 and RQ3, 
and was a source of secondary data as a precursor to interviews in Study B. Study 
B involved interviews with 15 learners with accessibility needs who had partici- 
pated in MOOCs and filled up the course-surveys. The data from the semi- 
structured interviews helped to understand their motivations (for RQ2), the 
accessibility barriers they found, whether/how they worked around the barriers, 
and their suggestions for desired solutions (R.Q3). It was essential to understand the 
individual situations learners had when working with MOOCs and to consider 
their varied contexts. Like in study A, thematic analysis was used for the analysis of 
the interviews, and survey data provided triangulation. 

Study C was developed to understand how to improve the accessibility in 
MOOCs (RQ3) from an expert evaluation perspective (Iniesto, McAndrew, 
Minocha, and Coughlan, 2019). The study employed an accessibility audit which 
was conducted on four MOOCs from FutureLearn, Coursera, edX and Canvas. 
The audit-instrument was developed as a part of the research programme. The 
audit-instrument was comprised of four main evaluation areas and, therefore, four 
different checklists to apply heuristic evaluations (Petrie & Bevan, 2009): 
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e Technical accessibility evaluation. Checking of conformance to guide- 
lines through WCAG (2018) and the text-based files. 

e User experience (UX) evaluation. The evaluation of usability and user 
experience characteristics of the user interface design and pedagogical 
design. 

e Quality evaluation. Evaluation of MOOC'S properties, the quality of the 
design, platform and support for learners. 

* Learning design evaluation. Evaluation of the learning design character- 
istics within MOOCs using Universal Design for Learning (UDL). 

(See Chapter 15, Iniesto & Hillaire, 2022) 


RQ3 was answered through three complementary studies. The findings from 
user-based studies were reinforced by the results from the audit which revealed 
further barriers. As a consequence of having different samples when merging 
research methods, the combination of methods allowed triangulation, to bring 
together complementary data interpretation and for checking validity (Creswell & 
Clark, 2017). 


9.4 Main findings across the three studies 


The combination of qualitative studies through interviews with MOOC provid- 
ers and learners, and the quantitative information provided by the MOOC survey 
data provided an in-depth and multi-faceted insight into the accessibility needs 
of MOCC learners. The MOOC accessibility audit helped to identify accessibil- 
ity barriers and the audit-instrument provides a tool that can be used and itera- 
tively developed further to support the design and evaluation of MOOCs for 
accessibility. 


9.4.1 MOOC providers cater for learners with accessibility 
needs (RQ!) 


There is an awareness amongst the MOOC providers of learners with accessibility 
needs participating in MOOCs. However, in the investigations in this research 
programme, the providers have acknowledged limitations. For example, MOOC 
providers often do not know who is participating in their MOOCs which leads to 
a lack of understanding of their learners, and in particular learners with accessibil- 
ity needs. MOOC providers do not gather accessibility information or require- 
ments from their learners as is typical of other educational environments (Porter, 
2014). Providers are, therefore, missing an opportunity to get more comprehensive 
feedback from learners to help them better support accessibility over time. 
MOOC providers agreed that the technology of the platforms is creating barri- 
ers. MOOCS use social media, third-party software and technologies that may not 
be accessible for all learners. Therefore, MOOC providers prioritise legislation 
over learners’ preferences and needs for MOOC accessibility. These factors have a 
direct influence in limiting the availability of accessible educational resources as 
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MOOC providers are not designing the educational resources for different target 
user-groups and are not allowing personalisation of the learner experience as a 
strategy to overcome accessibility barriers. 


9.4.2 The motivations of learners when taking part in MOOCs 
(RQ2) 


The motivations of learners with accessibility need to participate in MOOCs are 
broad and depend on factors already identified for MOOC learners (Ilgaz & 
Gulbahar, 2017; Serdyukov & Serdyukova, 2015). Learners with accessibility needs 
reported that they find MOOCs useful for personal development and continuing 
professional development, and as a route for access to higher education. As MOOC 
providers, they consider the low cost of MOOCs to be an important factor. But 
learners reported that some of these motivations are at risk since, low cost and 
openness are not a priority in the recent business models being adopted by MOOC 
providers (Ruipérez-Valiente, Martin, Reich, & Castro, 2020). Finally, MOOCs 
allow social interaction, which is facilitated by their massiveness and that learners 
can work from their preferred environment (for example at home with a laptop 
that includes assistive technologies). This flexibility helps learners in their self- 
regulated learning experience (Conde Gafaro, 2022). 


9.4.3 Making MOOCs more accessible (RQ3) 


The three studies of the research programme being reported have provided rich 
data, much of which aligns with other findings on accessibility and UX barriers in 
reported research on other (non- MOOC) online learning environments (Acosta 
& Luján-Mora, 2016). Those findings linked to technical aspects have also been 
highlighted by MOOC researchers (Akgul, 2018; Bong & Chen, 2016; Królak 
et al., 2017). In line with Straumsheim (2017), repeated identification of barriers 
indicates the slow speed in improving accessibility of educational technology and, 
hence, MOOCs; the same barriers are repeatedly identified, but solutions have not 
been provided as yet. 

The studies within this research have been innovative in terms of identifying 
quality and pedagogical accessibility barriers, which have not been comprehen- 
sively explored in previous research (Park, Jung, & Reeves, 2015). These barriers 
can be caused by the way MOOCS are limited to a specific time frame generating 
barriers for many learners who cannot follow the workload included each week in 
the courses. As reported, the term open is also creating friction within the stake- 
holders: often in current MOOC;, the entire educational content is not available 
from the beginning, or access to it is lost when the course is finished. 

Accessibility barriers in MOOCs can be found in several touchpoints of the 
learner journey, including the registration processes, search pages, information pro- 
vided before enrolling, in carrying out assignments or the use of discussions. 
Previous research has developed legislation, frameworks and services to address 
accessibility in MOOCSs (Sanchez-Gordon & Luján-Mora, 2017). However, 
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MOOC providers reported a limited ability to address barriers when MOOCs are 
being run. Learners indicated different ways they had responded to find ways to 
cope with accessibility barriers; these workarounds though were far from the 
desired solutions. The massiveness intended of MOOCs further implies a greater 
predisposition to be available, improvements in the help reporting services need to 
be in place in advance. 


9.5 Discussion and implications for the future 


Chapter 9 aimed to develop an understanding of stakeholders’ perspectives from 
both the producer and learner communities. Those aspects are aligned with the 
open world learning approach of this book by providing accessible educational 
resources and platforms to reduce the digital divide. This research reveals openness 
as an enabler against the new MOOC business models and “technical disruption” 
such as accessibility are restricting opening up education to a large/massive scale. 

Ferguson et al. (2018) describe a set of eight priority areas for MOOC develop- 
ment one of which is “wide access" built on accessibility and including those 
learners who are excluded from education. This research has reinforced the argu- 
ment that to achieve wide access other priority areas need to be achieved, such as: 
"develop appropriate pedagogies", "develop effective learning designs", "clarify 
learner expectations", “develop educator teams" (in accessibility), and “develop 
new approaches to assessment and accreditation". A holistic approach to increasing 
accessibility in those areas will help widen access to all learners. In line with the 
need for having a holistic approach of embracing accessibility in MOOCs dis- 
cussed in Rodrigo and Iniesto (2015), contributions from this research can lead to 
future research areas. MOOC accessibility research needs to consider technical and 
pedagogical aspects, and participatory approaches of including both MOOC pro- 
viders and learners in accessible MOOC design. 

Future research with learners may involve focussing on case studies with par- 
ticular accessibility needs in order to understand those needs in-depth. Such an 
approach would avoid medical models of clustering learners with accessibility 
needs and allow a "putting people and processes first" perspective (Cooper, Sloan, 
Kelly, & Lewthwaite, 2012). Possible other sources of data may also be considered 
such as surveys included in accessibility-related MOOCs and analysis of the activ- 
ity data of learners participating in MOOCs (Cooper, Ferguson, & Wolff, 2016). 
Another possible research area could be development of guidelines to support 
audit evaluations during the MOOC design and development processes (for plat- 
forms and educational resources) rather than post-implementation of MOOCs. 
These guidelines could be integrated in the process model for MOOC design. To 
help achieve such guidelines which are (and should be) iteratively developed to 
meet learner needs, participatory research methods of involving learners in the 
design processes should be considered (Toetenel & Bryan, 2015). 

As Ferguson, Sharples, and Beale (2015) suggest, MOOCS need to evolve to 
meet societal needs building on advances in technology, and so future research 
should continue developing accessibility profiling standards and their practical 
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applications in open education (Navarrete & Luján-Mora, 2018). As well as con- 
tinuously gathering feedback from learners, the requirements of learners evolve 
and, also, with the technological changes, there will be accessibility barriers which 
the learners may not have encountered before. Obtaining feedback from learners 
enables MOOC providers to adapt the platform designs and educational resources 
to meet the accessibility requirements of learners. Further research could also con- 
sider the role of learning analytics in addressing accessibility of MOOCs (Cooper, 
Ferguson, & Wolff, 2016) and aspects such as how learner-emotions affect learning 
(Hillaire, Iniesto, & Rienties, 2019). 

While universal design encourages active consideration of learners with accessi- 
bility needs (Iniesto & Hillaire, 2022), an extended approach should focus on 
designing for diversity. Learners have diverse needs, and, in practice, an approach that 
addresses diversity leads to a model of alternative solutions around a core learning 
design. This preferred design approach for diversity is also referred to as inclusive 
design; it seeks to augment a central design by adding in a consideration of particu- 
lar learner groups so that they are included, potentially through alternative design 
solutions rather than one design solution (Clarkson et al., 2013). Inclusive learning 
design aims to avoid the trap of looking at the technology that is needed for imple- 
mentation but instead considers learning design in terms of learning goals, a model 
of pedagogy, and pattern of interactions of those involved (Toetenel & Bryan, 2015). 
This approach is aligned to that implemented at The Open University in its aims to 
“strike the right balance between digital augmentation and the human element in 
providing accessible services” (McAndrew, Farrow, & Cooper, 2012, p. 16). 


9.5.1 Implications for practice 


While Chapter 9 has shown that there are inherent challenges in accessible MOOC 
production, they are integral to any online learning course production in an agile 
development methodology. As a general approach, online course providers should 
seek a better understanding of their learners and their needs. The interests of learn- 
ers with accessibility needs are varied and findings from previous research have not 
focused on accessibility. Any online course development processes need to be 
reviewed from the early design stages to produce accessible content, and the focus 
should change from meeting legislative requirements to actually meeting learners’ 
needs. Therefore, to make online courses more accessible, it is necessary to put in 
place processes to identify accessibility barriers, to strengthen mechanisms that 
involve the participation of learners in course design, and to facilitate agile response 
in addressing barriers. 
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Chapter 10 


Internet kiosks in Uganda 


A window of opportunities? 


Khadija Mohamud, Alison Buckler, Beck Pitt and 
Peter Twining 


10.1 Introduction 


Despite the massive expansion of education in Africa through the various global 
development frameworks in the last three decades, including the current Sustainable 
Development Goals (UN, 2015), several challenges remain which the international 
community has described as a "learning crisis" (World Bank, 2018). The evidence 
shows that African countries exhibit the highest population of out-of-school chil- 
dren for all school-age groups (UIS, 2019). Besides, data on the quality of schooling 
indicate that approximately 202 million children are not attaining minimum pro- 
ficiency in literacy and numeracy even after several years of attending school (Bold 
et al., 2017). The reasons for these low learning outcomes are multifaceted; how- 
ever, a fundamental determining factor is the lack of essential instructional resources 
(Figa et al., 2020). Looking at Uganda, the focus of this research, the government 
proposed a new education curriculum in 2018, formulated in 2019 to address the 
low learning outcomes (World Bank, 2019). However, this policy reform was not 
aligned with the development and distribution of instructional resources. Many 
schools lack basic infrastructure and teaching resources to support this new cur- 
riculum (Tromp and Datzberger, 2019). 

In addition to technical skills such as literacy and numeracy at the heart of the 
learning crisis narrative, the inclusion of a broad array of soft skills competencies, 
also known as 21st-century skills such as computer literacy, collaboration, confi- 
dence, communication skills, critical thinking, decision-making and problem- 
solving have been emphasised as an essential component of education in the 
rapidly changing labour market (Joynes et al., 2019). It has been outlined that 
these skills extend beyond the work environment as individuals translate the 
acquired knowledge into beneficial, practical action that impacts all areas of their 
lives (Reece and Reece, 2016), leading to an improvement in their quality of life 
(Joynes et al., 2019). Although many countries in Africa acknowledge the impor- 
tance of soft skills in their public policy (Care et al., 2016), limited evidence exists 
of how these skills have been integrated into the education curriculum in practice 
(Kim et al., 2019). 

Due to formal education deficiencies, non-formal learning opportunities have 
become a core component of knowledge acquisition and livelihood improvement 
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in Africa (Yasunaga, 2014). Information Communication Technology for 
Development has been associated with bridging these gaps (Jordan, 2020). However, 
many low-income communities in Africa cannot afford to access ICT on their 
own and therefore experience a digital divide, described as disparities in the access 
and use of ICT (Van Dijk, 2017). 

To bridge this digital divide, Community Technology Centres (CTCs), which 
are public-shared access to ICT resources with computers, Wi-Fi and, in some 
cases, mobile devices have been deployed in low-income contexts (Nemer, 2018b). 
Studies assessing CTCs in Brazil and Kenya indicated these facilities provide posi- 
tive benefits that enable users to access information on education, health, current 
affairs (Wamuyu, 2017) and provide a social space to address individual and com- 
munity needs (Nemer, 20182). 

Additionally, similar studies have focused on how CTCs could support educa- 
tional outcomes in the Global South (e.g., Dangwal et al., 2014; Mitra, 2014; 
Mohamud, 2016). However, the evidence showed most studies focused on the 
wider Global South with limited focus on Africa. Furthermore, although many 
young people are not attending school, as the data suggest, they may be involved in 
non-formal learning contributing to lifelong learning (Yasunaga, 2014). Chapter 
10 explores the impact of Internet Kiosks, a CTC intervention in Uganda on users' 
learning outcomes and influence on the quality of life. 

Moreover, studies on CTCs have often focused on reporting tangible positive 
contributions limiting the intangible contributions (Osman and Tanner, 2017; 
Tabassum et al., 2019), and scarce evidence exists on associated threats, particularly 
in the African context (Livingstone et al., 2017). This chapter presents a holistic 
assessment of impacts comprising three perceived influences identified by young 
people (in-school and out-of-school) and adults who were users of the Internet 
Kiosks in Uganda. 


10.2 Theoretical framework and methodology 


The theoretical framing in Chapter 10 is based on the view that individuals are 
embedded within their context and are influenced by the cultural and social prac- 
tices of the society they live in, drawing on (Vygotsky, 1978) sociocultural perspec- 
tive. It draws on knowledge construction as an enculturation process that 
incorporates tools as organising resources integrated into participants’ meaning- 
making processes. Tools include symbols and other devices (Lave, 1988) including 
ICT. Meanings are constructed through interpretation of situations and objects 
based on previous knowledge and experience (Zittoun and Brinkmann, 2012). 
This proposition provided a foundation for understanding how the research con- 
text influenced the Kiosks’ communities’ perception and determined how they 
engaged with them. 

This chapter’s findings were part of a larger research project conducted in 2018 
at two urban low-income suburbs in Uganda. Skills to Survive [pseudonym], a 
UK-based organisation, partnered with a local organisation and built two outdoor 
solar-powered Internet Kiosks in October 2015, each equipped with wireless 
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Internet and two computers with educational software. One Kiosk [Kiosk A] was 
located in a local primary school [Hillside school] and the other [Kiosk B] in a 
busy market area approximately six kilometres from the school. Each Kiosk had 
two sides: one side with a lowered computer screen intended for children and a 
higher screen on the opposite side intended for all other users. Chapter 10 focuses 
on the following research question: What are the perceived impacts of the Kiosks 
on users (young people and adults) at the two sites in Uganda? 


10.2.1 Methodology and analysis 


Chapter 10 adopted a qualitative case-study research methodology. A purposive 
sampling through gatekeepers and a snowballing technique were used to select 
participants who could provide information on this study's research objectives. 
A total of 50 participants (26 young people; eight adult users of the Kiosks, includ- 
ing four teachers; 13 young people' carers; and three young people's teachers) 
participated in this research. The data generated were drawn from twelve focus 
group discussions (FGDs) with young people (45—60 minutes per FGD), semi- 
structured interviews with all 50 participants (30-60 minutes per interview) and 
field notes. The focus groups and interviews were conducted in a quiet location 
familiar to participants and done in the language participants preferred (either 
English or Luganda, which the first author speaks). All the data were transcribed 
and analysed following Braun and Clarke’s (2006) thematic analysis approach on 
NViVo 12. Based on the analysis, quotes that are most representative of each theme 
were selected for presentation in this chapter. 


10.3 Findings 


Three main themes emerged from the thematic analysis encompassing perceived 
opportunities, perceived influence on users’ quality of life and threats. This section 
explores these sequentially. 


10.3.1 Perceived opportunities 


The data on perceived opportunities demonstrate how the Kiosks created a 
potential for users according to their interests and abilities, thereby providing 
them with relevant skills and knowledge that supported their activities. Users 
valued the Kiosks outlining, "It has helped us to nurture our talents..." (Jason, 
adult, Kiosk A) and described the changes that came in their life through their 
engagement. One out-of-school participant framed it as “... For me the [Kiosk] 
opened up a world that I would have never seen" (Suleiman,16, out-of-school 
boy, Kiosk A). The purposes of use that were key to facilitating opportunities were 
different for young people and adults and depended on whether users were 
enrolled in school/college or not. Users’ descriptions of the Kiosks’ perceived 
opportunities were grouped into three sub-themes: learning; instructional resource 
support; and computer literacy. 
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10.3.2 Learning 


The findings highlight that the Kiosks opened up new learning opportunities for 
all participants and is presented as follows: In-School Young people; Out of School 
Young People; and Adults. 


10.3.2.1 In-School Young People 


Young people who were enrolled in school felt the Kiosks supported them in 
learning a range of topics related to their school subjects. 


I have been able to expand my knowledge on various subjects. Instead of hav- 
ing to wait for the teacher to tell me everything in my school subjects... [...] 
the [Kiosk] has helped me to be ahead... and even when we have not been 
taught something but it is part of the syllabus for the term, I used to do research 
on it. In History... ‘Ngoni migration’; ‘Nyamwezi’; the history of Maasai’. ... 
chemistry, I learnt how to balance chemical equations through watching 
YouTube videos on balancing equations. ...Statistics, I watched videos that 
helped me learn how to find the class boundary, cumulative distribution func- 
tion, probability density function, linear regression, multiple regression... ... its 
simplified things for me. 

(Garry, 16, in-school boy Kiosk A) 


Most young people enrolled in school outlined the media resources facilitated bet- 
ter understanding of new concepts compared to what they were taught in class. 


Because on that computer we see them, but in the books, in the teacher's 
notes, they don’t show them to us. They show us only their functions and 
words but don’t show us their pictures. So, on that computer we see their 
pictures, functions and get more functions than in the books [teacher’s notes]. 

(Shaheen, 16, in-school girl, Kiosk A) 


The evidence particularly indicated that the Kiosks supported young people’s 
understanding of concepts in science and mathematics disciplines, explicitly out- 
lining how the digital resources stimulated their comprehension of topics in these 
disciplines. 


I wanted to learn and understand paper chromatography... I read on the topic 
separating mixtures on BBC bitesize and watched some videos on Khan 
Academy on this topic, the videos simplified things for me because I under- 
stood things better than in class. 

(Gift, 15, in-school girl, Kiosk A) 


Many participants reported that the research they conducted at the Kiosks’ com- 
puters had a positive influence on their overall school performance. 
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Actually it [the Kiosk] has helped me much because I was not performing well 
in Biology... But now the last two terms I got a D1 [Distinction 1 ranges 
from 85 to 100 percent] in Biology after searching on that computer. But 
before I used to get F9, C5 [lower grades as per the Ugandan education grad- 
ing system]. 

(Shaheen, 16, in-school girl, Kiosk A) 


Participants’ teachers also acknowledged that the Kiosks had a positive influence 
on students’ overall performance. For example, in the case of Shaheen above, her 
Biology teacher commented “...whatever she is doing has improved her perfor- 
mance...” (Mr. Williams, Shaheen’s teacher). 


10.3.2.2 Out of school young people 


Young people who were not enrolled in school noted the Kiosks supported them 
to learn concepts they felt were important to their daily activities. They also indi- 
cated using the Kiosks to engage with specific aspects of the school curriculum 
they felt were essential to supporting them with their tasks. 


I learnt farming methods that prevent soil erosion like contour farming and 
planting trees to break the wind. As you can see our land here is very hilly, in 
the rainy season I used to lose a lot of crops but now that has reduced. [...] 
I also learnt about some farming tools that I did not know about like dibber... 
Because I am a farmer, I needed to know maths so I can count when I am 
planting and harvesting and even when selling my fruits and vegetables. So, 
I went and started learning on that [Kiosk] addition and subtraction on Khan 
Academy. ... As I told you I stopped school some years ago, so I had forgotten 
what I learnt then. 

(Salim, 14, out-of-school boy, Kiosk B) 


Out of school participants also indicated the Kiosks supported them in attaining 
skills for specific interests they had. 


It has been helping me to learn about photography and photo editing. Now 
there is a photography shop near here that I volunteer at... 
(Suleiman, 16, out-of-school boy, Kiosk A) 


10.3.2.3 Adults 


The findings illustrate that adult users utilised the Kiosks to learn about entrepre- 
neurial concepts that facilitated enhancement of an existing business or stimulated 
innovative ways of income generation. For instance, Eddie noted the Kiosk pro- 
vided him with access to entrepreneurial information enabling him to establish a 
small-scale business. 
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For me I didn't have the courage to open up a business and I thought you 
need a lot of capital to open up a business. But one day... read a bit about 
small scale business, how to open up a small-scale business, what do I need? 
what do I have to know about it? and what should I focus on? ... I didnt 
think I was able to start up something... After I realised that, then I... got 
determined to open up this business. 

(Eddie, adult, Kiosk B) 


10.3.3 Instructional resource support 


As established in the context section, Kiosk A was located at Hillside school. 
Teachers felt the Kiosk supported instruction at the school by bridging its educa- 
tional resource constraint. 


I use it [Kiosk] when... we are learning about the skeleton because we don't 
have these physical parts ofa skeleton. ...we usually go to the [Kiosk] computer, 
we type in and different structures appear, then a child is able to identify how 
a skeleton is... 

(Ms. Elaine, teacher, Kiosk A) 


Ms. Jane, another teacher stated: 


...I take my kids to see what I teach physically at the [Kiosk], so it helps me 
to simplify my work as a teacher... 
(Ms. Jane, teacher, Kiosk A) 


Ms Jenna, another teacher explained the significance of the Kiosk in her lesson 
planning: 


Some books we use are not up to date, there are missing some things. So, when 
I need to teach kids something and I fail to get from the book they [the 
school] have given me, I go to the [Kiosk] I search for that thing. 

(Ms. Jenna, teacher, Kiosk A) 


By complementing teaching at Hillside school, the findings show Kiosk A posi- 
tively impacted the school’s reputation. 


...many people have picked interest of bringing their children to us. ...they 
know we have free internet; we can easily search and get to know what we don’t 
know, that means the academic standard is now improving than before. ...previ- 
ously if you come across a question I didn’t know, I just left it out and skipped 
it... I wouldnt give to the pupils because even as a teacher I don't know the 
answer. But now I can come across a question I do research on the [Kiosk] and 
once I get its explanation and understand, I teach it to my students. 

(Ms. Elaine, teacher, Kiosk A) 
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10.3.4 Computer Literacy 


Most young people underlined the Kiosks were their first interaction with com- 
puters and indicated learning basic operating systems functions, Microsoft Office, 
and Internet functioning. As a result, participants felt confident that they could 
now use the computer and Internet independently. 


...it has taught me so many things, I didn’t know how to use a computer but 
now I know, I have learnt how to use Microsoft word, PowerPoint and search- 
ing on the internet. 

(Ross, 15, out-of-school boy, Kiosk A) 


Some teachers also pointed out that Kiosk A was their first engagement with com- 
puters, providing them with the chance to develop computer literacy with the 
assistance of colleagues. 


I am one of the teachers who at first didn’t know how to use a computer 
totally. But since we had that computer, [Elaine, another teacher] has been 
helping me a lot in learning the computer because she learnt computer 
studies. 

(Ms. Jessica, teacher, Kiosk A) 


10.3.5 Perceived influence on users’ quality of life 


The second theme that emerged under opportunities was the influence on the 
quality of life, which explores both tangible impacts encompassing the Kiosks’ 
economic impact and intangible impacts that contributed to users’ overall well- 
being. These intangible impacts comprised of social impact, collaboration skills, 
leadership skills, problem-solving skills, increased motivation and self-belief and 
community learning beyond Kiosks’ users. The majority of stories and testimonials 
show that the access to ICT at the Kiosks provided users with the possibility to 
improve their quality of life through continuous knowledge access and connectiv- 
ity. Many participants felt they had evolved as a result of the knowledge they 
acquired. In the words of one participant: 


This would never have happened if the [Kiosk] was never there. So, the [Kiosk] 
was the main source of knowledge for me with everything I learnt because I 
started with the [Kiosk]. At that time, I didn't have even 1000 Uganda Shillings 
[$ 0.27] to buy data bundle to put on my phone. [...] I have been able to get 
jobs through social media... [...] It’s because the main source of knowledge 
was the [Kiosk]. If they repair the [Kiosk], I think you can get ten [Jasons] from 
the [Kiosk] because it has promoted me. 

(Jason, adult, Kiosk A) 


The findings indicate the Kiosks provided adult users with access to resources that 
enabled them to elevate their economic livelihood, leading to an improved quality 
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of life. For Eddie, the Kiosk built his capacity to have the confidence to start a 
small-scale business that grew to become a photography studio and electronic 
accessories shop. Jason, another adult user, noted that Kiosk A provided him with 
the opportunity to generate income from YouTube vlogging. 


...the [Kiosk] has helped us... I used my YouTube channel I post there my 
videos and get views and then they pay me... Last year they [YouTube] gave 
me 47,000 Uganda Shillings [$12]. 

(Jason, adult, Kiosk A) 


Some users identified that the Kiosks provided them with an opportunity to earn 
an income. In the case of Ms Jane, she described the changes that occurred in her 
life due to the financial assistance she obtained from the online connections she 
made at the Kiosk. 


... When I went, I opened Facebook and Gmail and I got friends who help 
me because they send me help from abroad even my kids have got sponsors 
which I think there are others who have benefited like me from the [Kiosk]. 
Before the [Kiosk] came I was so badly off girl what I am telling you, that’s the 
fact I was in a small house, I couldn't afford to pay for my kids’ school fees, but 
when those people came [Kiosk developers] and they opened for us Facebook 
so we got many successful friends abroad. 

(Ms. Jane, teacher, Kiosk A) 


Some users also noted gaining employment opportunities on social media via the 
Kiosk. 


It [the Kiosk] has helped me to get a job on Facebook, the other time when 
I was looking for a job everywhere, I found the job at the construction place 
up there on Facebook. They advertised it in a Facebook group. 

(Fuad, 17, out-of-school boy, Kiosk A) 


The findings show that the Kiosks became important spaces where users strength- 
ened social ties with their community through their interactions at the Kiosks. 
Many young people indicated they made new friends as a result of their engage- 
ment with the Kiosks. 


I had just moved here when the [Kiosk] was built. So, I was still new and 
through the [Kiosk] I made new friends. So, the [Kiosk] helped me to make 
new friends and to fit in, in this community. 

(Garry, 16, in-school boy, Kiosk A) 


In addition to the in-person interaction, most participants talked about the Kiosks 
offering them new opportunities to nurture relationships with people in other 
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geographical locations, nationally and internationally, creating a sense of connec- 
tion to a broader world. 


I found a way of communicating with my dad [in Abu Dhabi] at this [Kiosk], 
before I used to wait for him to call us once per week, but now I communicate 
with him directly through Facebook. 

(Gift, 15, in-school girl, Kiosk A) 


Adult users particularly emphasised speed and ease to which they were able to 
communicate improved their relationships with others. 


I've been connected to friends than before, so it has made my communication 
easy because sometimes mostly on WhatsApp... I could take a long time 
without loading airtime. But now with the WiFi of the [Kiosk] communica- 
tion has become fast and easy; I just communicate on WhatsApp with our 
supervisors... So, it has made communication easy for me. 

(Ms. Elaine, teacher, Kiosk A) 


Many young people appreciated the Kiosks’ collaborative nature, indicating that it 
encouraged them to develop collaborative skills. 


It makes me work together with other people and you show them the ques- 
tion or a problem... It promotes unity when you are working together because 
other children come and then you work together the question and then you 
get the answer, that's the opportunity I get from the [Kiosk]. 

(Patricia, 13, in-school Kiosk A) 


Furthermore, many young people identified a heightened feeling of accomplish- 
ment after achieving their goals and indicated that their motivation increased as a 
result. 


...like in my group at school they can say that you have the answer okay you 
give us, and it makes me feel good because I am becoming a leader. Because I 
know my group depends on me for answers, my interest in my studies increased 
and I do a lot of research. 

(Jacob, 14, in-school Kiosk B) 


Several participants also expressed becoming competent in what they had learned 
at the Kiosks, which suggested an improvement in their self-belief and 
confidence. 


I had a problem I didn't know anything, like I told you I dropped out in pri- 
mary three, but ever since they brought that computer, then I started using it 
for reading and all, now I can read, I can speak English. Now I know some- 
thing, there is a change on my life because of that computer. 

(Ross, 15, out-of-school boy, Kiosk A) 
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10.3.6 Perceived threats 


Despite the numerous opportunities, the Kiosks provided for all participants, a 
significant threat that emerged from the data analysis was the threat of children 
being exposed to pornographic content on the computers at the Kiosks. 


...those elder men watch blue movies [pornography] during the day when us 
children are there, and many children end up seeing things they should not be 
seeing. 

(Jacob, in-school boy, Kiosk B) 


Although the threat of pornography was present at both Kiosks, the data shows 
that this threat’s frequency was lower at Kiosk A than B, where a committee to 
oversee the Kiosk and the school established measures to mitigate it. 


So how we did to control that [pornography], we make sure for example for 
my case since I live here within the school, I make sure every morning... 
I could come here earlier than the pupils then I put it [the computer] on and 
I check what is there and remove anything that is inappropriate that a kid can 
come across. If it’s not there, I would just leave. So, this is how we are trying to 
control this. 

(Ms. Elaine, teacher, Kiosk A) 


10.4 Discussion and moving forwards 


Chapter 10 presents a holistic assessment of impacts comprising three perceived 
influences encompassing opportunities, the effect on the quality of life, and threats 
identified by users of the Kiosks in Uganda through the sociocultural lens. The 
findings reported in this chapter shows the Kiosks had numerous positive impacts 
on users. The evidence on learning demonstrates how the Kiosks supported young 
people enrolled in school to learn a range of school topics and illustrates how the 
media resources facilitated understanding of new concepts, particularly in science 
and mathematics disciplines. Many of them reported that the Kiosks positively 
influenced their overall performance, with some explicitly indicating improvement 
in their grades. These findings align with similar previous studies (such as Dangwal 
et al., 2014; Mohamud, 2016). These results contribute to this literature by present- 
ing evidence from an African context demonstrating explicit areas to which the 
Kiosks supported young people’s school learning. Teachers who participated in this 
research also outlined that Kiosk A helped bridge the educational resource gap at 
Hillside school, indicating they utilised the computers to demonstrate the concepts 
they taught and provided them with relevant teaching content. Overall, this evi- 
dence suggests that the Kiosks helped bridge some of the educational resource gaps 
outlined in this chapter’s introduction encountered in this research context. 

The findings further demonstrate how the Kiosks supported out-of-school 
young people to engage with school curriculum aspects that supported their daily 
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activities. This finding adds depth to the literature focusing on how ICT could be 
used in non-formal contexts (such as Yasunaga, 2014) to support out-of-school 
children. The quotes from the out-of-school participants demonstrate how a more 
nuanced approach can help them acquire relevant knowledge to navigate their 
everyday experiences. Additionally, the Kiosks provided adult users with entrepre- 
neurial knowledge that facilitated improvement in their livelihood. Most partici- 
pants also expressed that they had developed basic computer literacy through 
self-directed learning and assistance from peers, a finding that echoes similar CTC 
studies (such as Wamuyu, 2017). The data also shows that some users, particularly 
young people, shared the knowledge acquired with peers, teachers, and their fami- 
lies, suggesting that the Kiosks’ positive influence benefitted a wider community. 
Furthermore, adult users and out-of-school young people noted that the Kiosks 
provided them with access to resources that helped improve their economic liveli- 
hood. Some stated that the entrepreneurial concepts learnt at the Kiosks enhanced 
their existing business, and others highlighted accessing employment opportunities 
and resources that stimulated innovative ways of income generation, suggesting the 
Kiosks promoted improvement in users’ life quality. Many participants also reported 
that the Kiosks provided them with an opportunity to emerge as leaders as they 
assisted other users. Some participants also pointed out that the knowledge they 
acquired at the Kiosks provided them with the competence to implement what they 
had learned, suggesting that the Kiosks increased their confidence and self-belief. This 
evidence addresses the identified gap in the literature on soft skills development dis- 
cussed in the introduction of this chapter by demonstrating how non-formal learning 
contexts such as the Kiosks examined here begins to bridge this gap for low-income 
communities in Uganda. It also contributes to the anecdotal research on the intan- 
gible influences of CTCs (such as Osman and Tanner, 2017;Tabassum et al., 2019). 
The findings on social impact demonstrate that the Kiosks became vital social 
spaces where many users fostered social ties with their community through inter- 
actions at the Kiosks. Many young people linked this to the Kiosks’ collaborative 
nature, where they shared a computer and developed a peer learning process to 
address questions and assisted each other in using the computer. These findings 
contribute to the anecdotal data on social practices afforded by CTCs (such as 
Nemer, 2018a) by presenting evidence from an African context. It also shows how 
the Kiosks provided many users with an opportunity to build online connections 
and relationships that afforded them new learning opportunities, better communi- 
cation, and some of them obtained new income possibilities. This evidence sug- 
gests that the Kiosks contributed to the improvement of users’ quality of life. 
While the Kiosks provided numerous opportunities to the users in the low- 
income communities where they were built, the findings show that the threat of 
children encountering pornographic content on the computers due to some adults 
accessing them had a negative implication. This finding contributes to the litera- 
ture on children’s online risks (such as Livingstone et al., 2017) by providing evi- 
dence from a Global South perspective where data has been identified to be scarce. 
Although this threat was present at both Kiosks, the data shows a difference in 
impact at the two Kiosks as the school and committee overseeing the Kiosk 
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established a measure to mitigate this threat. This finding proposes a need for 
consideration on how this threat is mitigated when CTCs are being designed. 
Despite this threat that needs to be addressed, the findings demonstrate the Kiosks 
created opportunities for users in this low-income context. Although this research 
was limited to the specific low-income communities in Uganda, its findings have 
implication for stakeholders to inform implementation of CTCs in similar contexts. 


10.4.1 Implications for practice 


Chapter 10 provides insights into how CTCs could support education in low 
resource contexts and demonstrate how out-of-school young people can be sup- 
ported through non-formal learning to acquire relevant skills to help them navi- 
gate their everyday experiences. The Kiosks’ tangible and intangible impacts 
demonstrate how CTCs can support low-income communities in this context to 
achieve their desired goals and facilitated soft skills development. This evidence 
provides an insight into how underserved communities in Uganda and similar 
contexts can be supported through CTCs to create changes in quality of life. 
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Eliciting students' preferences for 
the use of their data for learning 
analytics 


A crowdsourcing approach 


Maina Korir, Sharon Slade, Wayne Holmes and 
Bart Rienties 


li.i Introduction 


Higher education institutions (HEIs) collect and use student data to improve oper- 
ations and course delivery (Siemens, 2013), for research purposes (Griffiths, 2017), 
and to improve teaching and learning in a process referred to as learning analytics 
(Long & Siemens, 2011). Examples of such uses are illustrated in Chapter 8 (Rizvi, 
Rienties, Kizilcec, & Rogaten, 2022) and Chapter 14 (Nguyen, 2022). With the 
growing shift to blended and online learning in higher education, virtual learning 
environments (VLEs) facilitate the collection of data about whether and how stu- 
dents interact with learning resources. VLEs are designed to record a vast amount 
of information about students’ behaviour, including number of clicks, time spent 
on the VLE, number of videos viewed, and number of forum posts (Rizvi, Rienties, 
Kizilcec, & Rogaten, 2022). This information may be used as a proxy for student 
engagement with a course, and to predict student success. Furthermore, the insights 
allow HEIs to improve educational practice in teaching and learning (Nguyen, 
2022). Student support teams can identify students thought likely to drop out or 
fail the course (Foster & Siddle, 2020; Herodotou et al., 2017). These predictions 
can be made sufficiently early to allow tutors to intervene and support students to 
improve their performance and outcomes. 

The institutional use of student data to facilitate various forms of student success 
has given rise to ethical and privacy concerns (Ferguson, 2012; Siemens, 2013; Slade 
& Prinsloo, 2013). Ethics in learning analytics may be understood as “the systemati- 
zation of correct and incorrect behaviour in virtual spaces” (Pardo & Siemens, 2014, 
p. 439). Ethical considerations focus on issues such as morality, student identity, and 
the institutions’ obligation to use student data (Slade & Prinsloo, 2013). Privacy in 
learning analytics may be understood as “the regulation of how personal digital 
information is being observed by the self or distributed to other observers” (Pardo & 
Siemens, 2014, p. 438). The value of privacy lies in its ability to promote relationships 
and autonomy, allowing people to limit what is known about them and to make 
decisions based on their values, without outside interference (Rubel & Jones, 2016). 

Empirical research has consistently demonstrated that students are often unaware 
of the use of their data for learning analytics (Jones et al., 2020; Roberts, Howell, 
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Seaman, & Gibson, 2016), and the student data their institution collects (Sun, 
Mhaidli, Watel, Brooks, & Schaub, 2019). When informed about potential uses of 
their data, students express varied responses: such as indicating a lack of concern 
about the use of their data in cases where the recipient and data uses are made clear 
(Vu, Adkins, & Henderson, 2019), and accepting institutional use of their data to 
benefit their learning (Slade, Prinsloo, & Khalil, 2019). At the same time, students 
also express concern, for instance, about being surveilled or tracked (Slade & 
Prinsloo, 2014). Consequently, there seem to be inconsistent perceptions of stu- 
dents and privacy concern in learning analytics. 

An area for further research, within the context of student privacy and learning 
analytics, 1s that of students! perceptions of the transactional nature of learning 
analytics (Ferguson, 2019; Wintrup, 2017). Students are asked (or are presumed) to 
consent to the use of their data for learning analytics so that data can be used for 
potentially beneficial purposes such as the provision of learning recommendations, 
or recommendations for remedial action and to improve student performance (Ho, 
2017; Siemens, 2013). Use of student data in these ways has potential for privacy 
harms, that is, possible injury to students through the collection and use of their 
data (MacCarthy, 2014). This includes loss of autonomy (Rubel & Jones, 2016), 
profiling, and identification of the individual whose data is used (Solove, 2009). 
While there is insightful research on students' perspectives of the ethics and privacy 
of learning analytics, little is known about students' perceptions of this risk/benefit 
trade-off and their preferences for the use of their data. Chapter 11 offers addi- 
tional insights in this context. 


11.1.1 Empirical research on students and privacy in learning 
analytics 


Findings from a number of studies converge on a common theme that students 
lack an awareness of learning analytics and about how their data is used for this 
purpose (Jones et al., 2020; Sun, Mhaidli, Watel, Brooks, & Schaub, 2019). In 
general, where they are informed about learning analytics, what data is used, 
and for what purpose, it might be argued that students appear positive about 
institutional use of their data to enhance their own and other students’ learning. 
This is based on data collected using semi-structured interviews with 112 
undergraduate students across eight universities in the USA (Jones et al., 2020). 
Other work, with a sample of students at a UK university, involving a survey 
(with 674 students) and focus group discussions (with 26 students) (Tsai, 
Whitelock-Wainwright, & Gašević, 2020) supports this perspective, as students 
in the focus groups indicated their support for institutional use of their data, 
but only for what they considered as legitimate purposes, namely, to comply 
with legal requirements, to improve educational services, and to improve the 
university's overall performance. It is noted that this positive perspective is con- 
ditional, thus, it is not clear whether negative perceptions of data use might 
arise in cases where there is insufficient institutional transparency surrounding 
use of student data. 
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One possible benefit of transparency about institutional use of student data is a 
reduction in privacy concerns as suggested by the work of Vu, Adkins, and 
Henderson (2019) who distributed a survey to 1,647 students at various HEIs in 
the USA. However, as previously stated, there are mixed results within the context 
of students’ privacy concerns about data use for learning analytics. In contrast to 
the findings of Vu, Adkins, and Henderson (2019), students in the study by 
Ifenthaler and Schumacher (2016) were willing to share data related to their stud- 
ies, but were less willing to share personal data or data trails collected from their 
use of aVLE. More specifically, of the 333 students who filled out the survey, 84% 
were willing to share course enrolment data, compared to 8% who agreed to share 
their medical data, and 9% who agreed to share their online user path for learning 
analytics purposes. 

The role of students’ acceptance of data use in exchange for learning-related 
benefits has been examined qualitatively in work by Tsai, Whitelock- Wainwright, 
and GaSevic (2020) and quantitatively in work by Slade, Prinsloo, and Khalil 
(2019). In the latter case, the authors indicate that 74% of the 215 study partici- 
pants stated that they were comfortable with the collection of their personal data 
in exchange for benefits such as personalised support. However, to the best of our 
knowledge, there is currently limited to no empirical research that has explored 
students’ perspectives of the privacy risks inherent in the use of their data for 
learning analytics. 

The privacy calculus theory and findings from related research (Dinev & Hart, 
2006; Laufer & Wolfe, 1977), suggest that there is a relationship between both per- 
ception of privacy risks and benefits of data use, and willingness to share personal 
information. Specifically, where there is a high perception of privacy risk, users are 
less willing to transact with their personal information (Dinev & Hart, 2006), 
whereas users expecting to receive benefits are observed to share more data (Li, 
Rathindra, & Xu, 2010). Therefore, the following research questions were identi- 
fied for Chapter 11: 


1 To what extent does an awareness of the possible privacy risks and/or the 
benefits of data use for learning analytics influence students’ data use 
preferences? 

2 What do students indicate as the motivation for their data use preferences? 


11.1.2 Methods 
1.1.2.1 Setting and participants 


Using the crowdsourcing platform Prolific, a sample was drawn from UK-based 
students. We sought to recruit an equal number of male and female participants. 
With respect to participants’ ages, research findings have demonstrated that older 
adults express higher levels of privacy concern than younger adults (Black, 
Setterfield, & Warren, 2018). Therefore, we recruited participants aged between 18 
and 25 years to enhance our evaluation of the influence of the interventions. A 
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total of 447 participants took part in the study. There were 216 male (48.3%) and 
231 female (51.796) participants. The mean age was 20.6 (SD — 1.86). Most of the 
participants (409—91.596) were studying at university and the remainder were in 
further education (38-8.5%). 


11.1.3 Study materials 


All participants were shown a sample learning analytics dashboard (Figure 11.1), a 
data use preference prototype (Figure 11.2), and the privacy risks and/or benefits 
interventions (Figure 11.3). The latter was not provided to participants in the con- 
trol group. The design of the sample learning analytics dashboard was based on the 
OU Analyse interface (Kuzilek, Hlosta, Herrmannova, Zdrahal, & Wolff, 2015) and 
was simplified to maintain participants' focus on the study aims. 

The data use preference prototype showed participants two types of data that 
can be used for learning analytics, specifically data about the student and data 
about the students' activities on the online learning platform (Sclater, Peasgood, & 
Mullan, 2016). 

The privacy risks intervention was developed using Solove's (2009) taxonomy 
of privacy harms. The first risk (1) is referred to at the beginning and end of the 
description. It relates to the information collection category of the taxonomy and 
the risk of surveillance. The second risk (2) falls under the information processing 
category of the taxonomy, and the risk of aggregation. The third risk (3) is also in 
the information processing category of the taxonomy, under the risk of identifica- 
tion. Additionally, the benefits intervention presented nudging, prediction, and rec- 
ommendation of learning resources as benefits for students based on the use of 
their data. 


r Learning Analytics Dashboard 


r— Performance 


TD Prediction [Actual | | This section shows that you 
are predicted to submit all 
Submit 288 our required assignments 


2 Submit 74 for this course. 

3 Submit Not yet available This section contains 
personalised study 

4 Submit Not yet available recommendations. 


— Personalised Reading and Activity Récsmimendatont 


S23 Y 
Review course Access relevant Access practice 
concepts e-books assignments 


Figure 11.1 The sample learning analytics dashboard. 
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r— Data about myself, for example: 


* Age 

* Gender 

* Previous education 

* Number of course attempts 


r— Data about my activity in the virtual learning — 
environment, for example: 


* Activity on the student forum e.g. 
forum posts 

* Activity interacting with the course 
content e.g. downloads 

* Interaction with course material e.g. 
reading text 


Figure | |.2 Examples of student data used for learning analytics. 


Possible privacy risks of data use 


We will monitor what you and other students are doing on the online learning platform [1]. Data that 
you and other students have provided to separate information systems at your learning institution 
(for example during registration) will be combined to form a digital profile [2]. The digital profile can 
be linked to the individual student [3], and this information will be used to make decisions about you 
and other students, such as predicting your performance and giving you study recommendations [1]. 


Possible benefits of data use 


We can offer you personalised support to help you complete the course, including nudging to submit 
assignments or follow up from the student support team. We can also provide you with personalised 
recommendations of learning materials that can improve your understanding of the course material. 


Figure | |.3 Descriptions of the privacy risks and benefits. 


11.1.4 Study measures 


Three measures of data use preferences, concern over data use, and concern over 
privacy risks were created. It was necessary to create these three measures as there 
was limited research on students’ data use preferences in the learning analytics 
context, and therefore there were few opportunities to identify questions from 
related research as recommended in best practice for questionnaire design (Bryman, 
2016; Groves et al., 2009). 

Four other study measures were obtained from published research. The scale 
perceived usefulness of learning analytics was adapted from Arbaugh (2000) who 
developed it with 114 students in a study on student satisfaction with MBA courses. 
The sharing data scale was developed with over 300 students in Germany (Ifenthaler 
& Schumacher, 2016). The scale perception of benefit from data use for learning 
analytics was adapted from Naeini et al. (2017) who used it with 1,014 participants 
in a study on privacy preferences in the Internet of Things. Finally, the Internet 


Eliciting students’ preferences 149 


Users Information Privacy Concern (IUIPC) scale (Malhotra, Kim, & Agarwal, 
2004) was developed in two studies with over 700 participants and has been used 
extensively to measure users’ privacy concerns. The scales used in the study were 
modified to include a “not applicable" option following recommendations by 
Aldridge and Levine (2001) and Krosnick (2018) to allow participants to respond 
even if a question did not apply to them. Additionally, attention check questions 
were used to ensure that spurious data could be detected in the data cleaning phase 
(Egelman, Chi, & Dow, 2014). 


11.1.5 Study design and procedures 


A between-subjects design was used where each participant was randomly assigned 
to one of four groups: the risks group, the benefits group, the risks and benefits 
groups, and the control group. After providing consent to take part in the study, 
participants indicated their data use preference (pre-test), choosing between pre- 
ferring to share no data, only data about themselves, only data about their activities 
on the learning platform, or both data about themselves and their activities on the 
learning platform. They were given brief background information on learning 
analytics and then viewed the sample learning analytics dashboard. In the experi- 
mental condition participants were shown the intervention, and afterwards they 
indicated their level of concern for the stated privacy risks and their perception of 
the benefits. Participants were then asked to assess the usefulness of the learning 
analytics dashboard features and indicate whether they were concerned about the 
use of their data. They again provided their data use preferences (post-test) and 
indicated their general privacy concern, before providing demographic informa- 
tion at the end of the study. 


11.2 Results 


11.2.1 The influence of risks and benefits awareness on 
participants’ data use preferences 


In terms of RQ1, the descriptive statistics for participants’ data use preferences in 
terms of the mean and standard deviation are shown in Table 11.1. There was a 
decrease in the mean values (post-test) for the control group and the risks group, 
and an increase in the mean values for the benefits group. At the same time, the 
mean values for the risks and benefits group remained unchanged. In other words, 
the results suggest that the awareness intervention might have had an influence on 
participants’ data use preferences in the risks, and in the benefits group, but made 
no difference in the risks and benefits group. It might be that any increase in par- 
ticipants’ data use preferences (thereby indicating a willingness to share more data) 
resulting from the benefits intervention was tempered by the risks intervention. 
There was a slight decrease comparing the overall post-test and pre-test mean 
scores (pre-test mean = 3.03, SD = 0.90; post-test mean = 3.00, SD = 0.94). A 
paired samples t-test revealed that these differences were not statistically significant 
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Table II.1 Descriptive statistics of students’ pre-test and post-test data use 
preferences by experimental group 


Data use preference Data use preference 

pre-test post-test 
Condition Mean Std. Deviation Mean Std. Deviation N 
Control 3.09 0.976 2.97 0.98 128 
Risks 2.93 0.906 2.89 0.934 104 
Benefits 3.04 0.858 3.07 0.906 104 
Risks and 3.05 0.824 3.05 0.923 EN 


benefits 


(p > .340). A one-way ANOVA revealed no significant differences among the 
means of the four groups on pre-test data use preferences (F(3, 443) = 0.637, p > 
.590), and post-test data use preferences (F(3, 443) = 0.786, p > .501). Finally, using 
McNemars test, as the variables were at the nominal measurement level, we deter- 
mined that there was no statistically significant difference in participants! data use 
preferences pre- and post-intervention (p > —.140). 


11.2.2 Motivation for participants’ data use preferences 
11.2.2.1 Theme l: Support for institutional use of student data 


Two main themes were identified from participants’ open responses in order to 
address R.Q2. The first theme indicated participants’ support for institutional use of 
student data (4996 of codes, n — 238), and participants gave several reasons for their 
data use preferences (8096 of codes, n — 190 (out of 238 codes)), for example, that 
the data shared was sufficient or appropriate for the stated purposes (19% of codes, 
n — 37 (out of 190 codes)). Their perception of the data being sufficient took on 
several forms, for example, they shared what was most relevant (38% of codes, n = 14 
(out of 190 codes)), was less invasive (19% of codes, n = 7 (out of 190 codes)), felt 
comfortable or safe for them to share (1996 of codes, n — 7 (out of 190 codes)), or 
what they thought showed their engagement (1196 of codes, n — 4 (out of 190 
codes)). As participant 161 stated: 


Because that is directly related to my learning and doesn’t take into consider- 
ation other factors which may not assess academic performance. 

(P161, risks group, Female, willing to share data about 

activities, no change in data use preference) 


This code suggests that the way student data will be used is a useful information 
point in transparency initiatives as students might relate to one or more of the 
stated purposes, thereby agreeing to the use of their data, as seen in (Slade & 
Prinsloo, 2014). 
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Participants additionally expressed several expectations of what the learning 
institution should do with their data (996 of codes, n — 21 (out of 238 codes)). 
For example, they expressed an expectation for purpose limitation (3396 of codes, 
n — 7 (out of 21 codes), that is, that only academic data would be used for 
academic purposes: 


I do not think it is appropriate to use data about a student's private life and 
background to make a judgement on their academic performance. It is not fair 
to do so, as it could lead to discrimination and unfair bias. A student's academic 
performance and private life should be separate and it is not the place of the 
university to be able to access that data or use it to judge a person’s abilities. 
Their abilities should be judged solely on their present engagement with the 
course and their previous academic record. 

(P269, control group, Female, willing to share data 

about activities, no change in data use preference) 


Here we see the role that context plays in students’ expectations of institutional 
data use. In contextual integrity (Nissenbaum, 2004), there are generally expecta- 
tions around what information about a person can and cannot be revealed in a 
given context. 

Finally, participants were observed to make trade-offs in data use for benefits 
even while supporting the use of student data (4% of codes, n = 9 (out of 238 
codes)). For instance, participants indicated that they had shared just enough to 
protect privacy (4496 of codes, n — 4 (out of 9 codes)), that they sought the best 
balance between privacy and services for students (22% of codes, n = 2 (out of 
9 codes)), they were getting something back for their information (1196 of codes, 
n = 1 (out of 9 codes)), and that the benefits outweighed the privacy risks (11% 
of codes, n = 1 (out of 9 codes)). 


11.2.2.2 Theme 2: Hesitation about institutional use of student data 


The second theme highlighted participants’ hesitation about institutional use of 
student data (51% of codes, n = 247). Participants provided various reasons why 
they hesitated to share (all) their data. These reasons clarified why they chose to 
share some data, that is data about themselves or data about their activities (43% of 
codes, n = 104 (out of 247 codes)). One reason that participants agreed on was that 
personal details were either not needed or should not be shared (42% of codes, n 
= 44 (out of 104 codes)). A preference for privacy (23% of codes, n = 24 (out of 
104 codes)) was another reason why participants hesitated to share their data, as 
participant 64 stated: 


I tend to avoid giving away personal information as I like to be private. 

Information about what I do on my university’s learning platform is ok 
though. 

(P64, risks group, Male, willing to share data about 

activities, no change in data use preference) 


152 Maina Korir et al. 


As explained previously, the context influenced the participants data use prefer- 
ence, helping him make an exception because it was the university learning 
platform. 

Furthermore, participants raised ethical and privacy considerations (3596 of 
codes, n = 87 (out of 247 codes)). Their responses captured their concern about 
(potential) bias, discrimination, or prejudice (2196 of codes, n — 18 (out of 87 


codes)). 


I feel the knowledge of certain things such as my gender may be used to 
discriminate. 

(P412, risks group, Female, not willing to share data, 

change to prefer to share no data) 


The example above demonstrates that transparency initiatives may cause stu- 
dents, where they can control whether their data is used, to prefer not to share 
any data. As this participant was in the risks group, the change in their data use 
preference was unsurprising. Transparency initiatives in learning institutions 
should seek to balance information about privacy risks alongside information 
about benefits, thereby enabling students to make informed decisions about the 
use of their data. 

Additionally, participants raised concerns that institutional use of student data as 
described in the study could negatively impact students (18% of codes, n = 16 (out 
of 87 codes)), for instance, that students would be pressured to behave in a certain 
way: 


With more information, I could determine that the personal information used 

would be almost a breach of my privacy, and even giving away data about my 

use of the learning platform is somewhat private to me, as I would like to 

privately access learning materials without feeling pressure (for example if 
I downloaded some materials a little late in the course, or past a deadline). 

(P425, control group, Male, willing to share data about 

activities, change to prefer to share no data) 


Finally, other concerns were raised including that the data use was privacy invasive 
(8% of codes, n = 7 (out of 87 codes)), and that the data could only give a partial 
picture of the student (8% of codes, n = 7 (out of 87 codes)). 

There was some tension observed between understanding the need for data use 
and discomfort with data use (12% of codes, n = 29 (out of 247 codes)) where 
participants appeared in two minds about the use of data. Participants were seen to 
express an understanding, for instance, that institutional data use was needed, 
alongside seemingly contradictory views, such as expressing corresponding con- 
cerns about discrimination, or a sense that the data use was privacy invasive: 


I don’t mind giving basic information about myself since that would be fairly 
easy to get anyway, but I do not like to have everything about me being 
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tracked even it could have some minor benefits to helping me improve my 
performance. 

(P424, Risks and benefits group, Female, willing to share 

data about self, no change in data use preference) 


Finally, participants expressed a desire for boundaries or separation in data use 
across their personal lives and their lives as students (8% of codes, n = 19 (out of 
247 codes)). They were keen to keep academic and private life separate or their 
online activity separate from student life: 


At first I thought it might be a good idea to share some data, but I believe that 
the suggested options of the data shared/what will be done with it oversteps 
its boundaries and could have negative effects on performance and mental 
health. I believe that if the only outcome of the data collection was to improve 
learning by providing support, then I'd be alright with sharing some of the 
suggested data. 
(P326, control group, Female, willing to share data about 
self and activities, change to prefer to share no data) 


11.3 Discussion and moving forwards 


In Chapter 11 participants were presented with privacy risks and/or benefits inter- 
ventions to examine whether and how these would influence their data use prefer- 
ences. While we observed slight changes to participants’ data use preferences, these 
changes were not statistically significant (RQ1). Therefore, we analysed partici- 
pants’ open responses to better understand motivations for their data use prefer- 
ences (RQ2). 

We identified nuances in participants’ responses as they expressed support for 
institutional use of student data for learning analytics alongside hesitation to sup- 
port institutional use of student data. While one would expect either full support 
for use of student data or complete refusal to support the same, participants’ 
responses suggested a middle ground where this apparent tension between support 
and hesitation co-existed. 

Participants’ responses indicated that they made trade-offs to arrive at what was 
an acceptable use of student data for them.This suggests a hidden negotiation pro- 
cess that students go through. Learning institutions can provide supporting struc- 
tures such as inviting and publicly responding to students’ questions on institutional 
data use to make these tensions and negotiations visible. There are also different 
student preferences to consider and support. While some students might want to 
choose what data is used, others may find this effort a step too far. However, this 
apparent apathy should not be construed as students lacking an interest in or hav- 
ing no concerns over the privacy of their data (Hargittai & Marwick, 2016). 

Throughout Chapter 11 we noted that participants had contrasting views on 
what data was appropriate to share and why. For example, one student shared data 
about themselves saying that was less invasive, while another student shared data 
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about their activities on the online learning platform for the same reason. This sug- 
gests a need to enhance students' data literacy. For instance, they may not know that 
their personal data has less prominence in the statistical models over time and data 
about their activities on the learning platform becomes more important (Kuzilek, 
Hlosta, Herrmannova, Zdrahal, & Wolff, 2015). Additionally, it may be unclear 
whether sharing different data modifies the digital profiles created about students, 
how the resulting digital profile influences the benefits available to students, and 
the corresponding privacy harms. In this way, students can make more informed 
decisions about the use of their data which should be an aim of learning institu- 
tions' transparency initiatives. 


11.3.1 Implications for practice 


We recommend greater transparency from learning institutions about institutional 
uses of student data. This would require that the relevant content is made accessible 
and understandable for students, identifying what and how specific data is used for 
learning analytics purposes. This level of detail in learning institutions’ transparency 
initiatives will be received positively by some students. Teachers can also support 
institutional efforts for transparency around data use by making students aware of 
when and how their course data is used for learning analytics. Furthermore, insti- 
tutions should examine ways to empower students with respect to the use of their 
data by allowing them to indicate whether they want to participate in learning 
analytics, and which data items they would be willing to have used for the same. 
Whatever students choose, ethical practice places a burden on the learning institu- 
tion to ensure that the benefits truly outweigh any harms. 
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Chapter 12 


Measuring player creativity in 
digital entertainment games using 
the Creativity in Gaming Scale 


Johanna Hall, Christothea Herodotou and 
loanna lacovides 


12.1 Introduction 


In 2020, the global population of gamers stood at 2.7 billion with an increase of 
over 6% globally from the previous year. Breaking this down further, those who 
play games have increased in all regions of the world, with the largest increase of 
gamers being in the Middle East and Africa, with a 14% increase from the previous 
year. This growth is even more prominent with the rise of smartphone usage and 
which is now the largest gaming platform in global terms (Newzoo, 2020). The 
increasing ubiquity of digital games, now widely available on a variety of different 
mediums including mobile, PC, console and streaming services, has meant that the 
benefits of gaming are reaching an even wider, more diverse demographic. 

Digital games have been argued to be an effective medium to foster creativity 
as they encourage players to overcome challenging problems and actively experi- 
ment with different ideas and solutions free from external constraints (Leng et al., 
2010). Games often comprise ill-structured challenges (e.g., problems with more 
than one solution) which have been argued to facilitate intrinsically motivating 
flow experiences leading to creativity (Kiili, 2005). The relationship between 
creativity and gameplay has been investigated by studies such as Moffat et al. 
(2017) who found that playing games contributes to a creative state of mind, and 
Blanco-Herrera et al. (2019) who identified a positive correlation between 
gameplay and trait creativity scores. 

While general conceptualisations of creativity attempt to account for the many 
facets of, what has been argued to be, a multidimensional construct (Shute & 
Wang, 2016), they do not provide information on how creativity may manifest 
specifically within digital games. By understanding these unique manifestations of 
creativity, digital games may be used as an alternative, widely available tools for 
open-world learning. In this way, not only educators may select different games to 
develop creativity in students, but learners will have the autonomy to identify 
areas of creativity they wish to develop and select games according to their 
preferences. 
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12.1.1 Creativity in digital games 


Hall et al. (20202) attempted to classify the different ways creativity is expressed in 
games by identifying three main categories: creativity as problem-solving, creativity as 
appropriation and creativity as affective change. Creativity as problem-solving relates to the 
unique solutions and approaches to gaming problems that players create, and is 
likened to “Little C" (Kaufman & Beghetto, 2009) or "Everyday creativity” 
(Richards, 2007) in that the creative solutions/ideas are unique to the individual 
who creates them. Work such as Kiili’s (2005) Experiential Gaming Model extrap- 
olated how games facilitate creative problem-solving through an experiential pro- 
cess where players constantly testing, reflecting on, and refining solutions. Similarly, 
Iacovides et al. (2014) illustrated the unique approaches players create to navigating 
gameplay breakdowns, such as taking a break or using a trial and error approach. 

Creativity as appropriation relates to the ways players go above and beyond what 
developers expect, personalising the gameplay experience (Herodotou et al., 2012). 
Appropriation includes emergent (Jarrett, 2014) and transgressive play (Aarseth, 
2007) where players can combine game variables and mechanics in unintended 
ways, use glitches and create additional challenges such as speedrunning (i.e., com- 
pleting the game in as short a time as possible). Furthermore, appropriation also 
includes transformative play (Sotamaa, 2007) including the creation and use of 
modifications, and user-created content (Burri, 2011) within the game such as 
object creation, and around the game such as fanfiction and fanart. 

Creativity as affective change is most synonymous with Kaufman and Beghetto’s 
(2009) “Mini C" whereby creativity is conceptualised as a players unique interpre- 
tation of gameplay and game narrative which can lead to “reflection on the game’s 
narrative, and as a result, the alteration of thought patterns, beliefs and perspectives” 
(Hall et al., 2020a, p. 9). Work such as Bopp et al. (2018) has investigated how games 
can instigate affective change, including reflection (Mekler et al., 2018), through 
emotional challenges which are presented through choice-based dialogues and 
ambiguous narratives, and perspective-challenging moments where players’ prior 
concepts, attitudes and expectations are challenged (Whitby et al., 2019). 


12.1.2 Measures of creativity 


Creativity has been argued to be a hard-to-measure construct due to a variety of 
factors, such as a lack of clear definition and operationalisation, theoretical multi- 
dimensionality, and difficulty separating trait from state (e.g., some people may only 
be creative in certain situations, while others may be generally creative) (Shute & 
Wang, 2016). As such, there exist a number of metrics that aim to assess creativity 
from a variety of perspectives. 

For example, some psychometric assessments rely on divergent thinking which 
has been “conceptualised as one component of creative thinking [and] remains an 
important concept among creativity researchers” (Baer, 2016, p. 9). The most well- 
known divergent thinking test is the Torrance Tests of Creative Thinking (TTCT) 
(Torrance, 1966) which measures creative attributes such as fluency (total number 
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of ideas), flexibility (number of different categories or kinds of ideas), originality 
(distinctiveness of an idea), and elaboration (richness of detail and broadness of 
scope of ideas) (Baer, 2016). However, while widely used, TTCT tests have been 
criticised for measuring only one aspect of a multi-faceted construct, and being 
too closely linked to intelligence (Lubart, 1994). Other psychometric approaches 
such as the Remote Association Test (RAT) (Mednick, 1968) assess an individual's 
ability to form associations and combinations between diverse knowledge ele- 
ments, and involves a series of three words indirectly related to each other which 
the participant must connect with a fourth word (Lubart, 1994). The RAT has 
been criticised for having poor content validity by measuring verbal ability and 
academic performance instead of creativity (Lubart, 1994). 

Other metrics aim to measure creativity based on personality factors such as the 
Creative Personality Scale (CPS) (Gough, 1979) and the Creativity Style 
Questionnaire (Kumar et al., 1997). Biographical inventories assess creative talent 
via observational measures by instructors, supervisors, or classmates to assess behav- 
iour and compare it to creative outputs and sensitivity such as the Creative Behavior 
Inventory (CBI) (Kirschenbaum, 1989). Measurements of creative products include 
metrics such as the Creative Product Assessment Matrix (Besemer & O’Quin, 
1987) which attempts to analyse how creative a product/idea is. 

While far less common, there are some examples of measurements of creativity 
for digital games. For example, Chuang et al. (2015) documented the creation of 
an objective metric for measuring creativity in the game Arctic Quest 2. Using a 
computerised fuzzy logic system that extended evaluations beyond true/false and 
good/bad, player’s trophies and points were recorded and used to calculate scores 
on fluency, flexibility, elaboration and sensibility. While Chuang et al.’s (2015) com- 
puterised method eliminated some of the subjectivity in creativity assessment, it 
does not measure originality (e.g., novelty and relevance of ideas) which has been 
argued to be a key component of creativity (Runco & Jaeger, 2012). Furthermore, 
it is based on only one game, and does not account for the specific manifestations 
of creativity in digital games, instead of using the psychometric properties of gen- 
eral creativity metrics. 

Huang and Chuang (2012) devised the Creativity Assessment Scale of Digital 
Game Story Design (CAS-DGSD) to measure the creativity involved in creating 
game narrative. The CAS-DGSD includes the creativity dimensions of novelty, 
complexity, imagination, variation and application, and the digital game story con- 
structs of content, narrative device, organisation, word choice and game feature. 
While the CAS-DGSD provides a valuable assessment tool for evaluating game 
narrative, it is geared more towards the creativity involved in designing a game, 
rather than the creativity involved in playing one. 

In essence, a range of metrics exist to measure creativity from a variety of theo- 
retical standpoints; however, the majority of these do not pertain to the unique 
expressions of creativity in digital games. The scope of creative practices which 
games facilitate may be difficult to quantify with traditional measures of creativity 
such as the TTCT (Torrance, 1993) and the CPS (Gough, 1979), and work which 
has attempted to create metrics for the assessment of creativity in digital games 
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such as Chuang et als (2015) fuzzy logic system focused on one game genre using 
existing psychometric constructs to measure creativity. As such, this chapter details 
the development of a scale to measure creativity unique to digital games and is not 
based on existing psychometric or traditional measures. The scale operationalises 
creativity specific to digital games and provides a measure of the unique expres- 
sions of creativity and associated constructs from the ground up. 


12.2 Method 
12.2.1 Research instrument 


The instrument for data collection was an online survey, which consisted of three 
parts with the first comprising the consent questionnaire. The second part asked 
respondents about their current gaming habits, including how many hours a week 
they spent playing online and offline games, genre of games played, format of gam- 
ing (e.g., PC, console, handheld) and what type of gamer they identified as, if any 
(e.g., casual, moderate, hardcore). The third part of the survey comprised a total of 
71 attitude statements across seven sections. The statements related to themes 
developed from a previous qualitative study (discussed in Hall et al., 2020a, 2020b) 
which identified three unique expressions of creativity in digital games using a 
thematic approach; problem-solving, appropriation and affective change, in addi- 
tion to further related themes including player conceptualisations of creativity, 
design affordances for creativity, learning from creativity and transferability of cre- 
ativity. As such, three sections related to the different forms of creative expression, 
one related to how players conceptualised creativity, one related to learning out- 
comes from creativity, one related to the transferability of game-based creativity 
and one related to creative design affordances in games. Scale items were on a 
standard 5-point Likert scale, with 1 being strongly disagree and 5 being strongly 
agree. At least one negatively worded item was included in every Likert section. 


12.2.2 Recruitment 


The study advert was posted on a variety of online forums, social media and 
Reddit. A wide selection of gaming forums was chosen to capture multiple differ- 
ent game genres and, when available, the advert was placed in sub-forums such as 
off-topic (non-game related) and community creation areas. Forums included the 
popular platform Steam, in addition to game-specific forums such as Kerbal Space 
Program and Bioware. Adverts were also placed on speedrunning and game modi- 
fication forums such as Nexus Mods. Reddit was used to target specific games 
from a variety of genres (e.g., r/ggoverwatch, r/witcher3) and groups (e.g., r/ 
gamedev, r/gamingpc) and the survey specific sub-reddit r/samplesize was also 
used. The advert was posted on the principal researcher Facebook and was shared 
on Twitter using hashtags such as #videogamers and #gamedevs. Due to the high 
proportion of males in the sample, Twitter was also used to target female gaming 
and tech groups such as #womenintech and #girlgamers. 
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12.2.3 Participants and data analysis 


A total of 251 respondents completed the survey, comprising 160 males, 82 females, 
6 nonbinary, and 3 undisclosed. The most common age bracket was 25-34 (n = 
108), followed by 18-24 (n = 89). The sample consisted of predominantly American 
and European nationalities and was fairly even across education levels, with the 
most common being undergraduate and college (high school — 39, college — 60, 
undergraduate — 81, postgraduate — 40, doctorate — 18, and other — 10). 

The stage of analysis comprised dimension reduction using exploratory Principal 
Component Analysis (PCA) with Varimax rotation in order to "summarize data so 
that relationships and patterns can be easily understood [and] to regroup variables 
into a limited set of clusters based on shared variance" (Gie Yong & Pearce, 2013, 
p. 79). The higher cut-off point of.45 which is described as fair to good was 
adopted (Tabachnick & Fidell, 2007). All of cross-loading items were also removed. 


12.3 Results 
12.3.1 Descriptive statistics 


Respondents played a wide variety of games with the most common online genres 
being massively multiplayer online role-playing games (MMORPG) (n = 127) and 
massively multiplayer online first-person shooters (MMOFPS) (n = 106), and the 
most common offline game genres being role playing (n = 214), action/adventure 
(n = 187), strategy/tactics (n = 146) and first-person shooter (n = 124).The mean 
hours spent playing online games a week was 12.2 (SD = 14.4) and the mean hours 
spent per week playing offline games was 14.0 (SD = 12.3).The mean gaming ses- 
sion lasted 3.4 (SD = 2.3) hours. The most common gaming medium was PC/ 
Mac/Linux (n = 222) followed by Console (n = 133). 


12.3.2 Factor analysis 


Items were subjected to PCA using an iterative process whereby all items which 
loaded under .45 and cross-loaded were removed before PCA was rerun until no 
items cross-loaded or loaded under .45. The resulting initial analysis of the 71 scale 
items produced a twenty-factor model accounting for 68% of variance, with a 
Kaiser-Meyer-Olkin value of .814 and a significant result for Bartlett’s test of 
Sphericity (X? = 7555.407, DF = 2485, p < 0.001) indicating that a reasonable 
factor solution could be computed. Upon analysis of the scree plot, it was evident 
that five- and six-factor solutions could be supported (Figure 12.1). 

Both five- and six-factor solutions were attempted, with the difference of vari- 
ance between them being 1% (55% and 56%, respectively). Due to the minimal 
variance difference and due to the five-factor solution providing ease of identifica- 
tion of the constructs within the data, it was selected as the model of choice. 

The five-factor model was subjected to the same iterative process as the original 
factor model whereby all items with loadings under .45 and cross-loadings were 
removed and PCA was rerun until no items cross-loaded or loaded under.45.The 
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Figure 12.1 Scree plot of initial twenty-factor mode. 


resulting model accounted for 57.4% of variance. The Kaiser-Meyer-Olkin mea- 
sure of sampling adequacy was good at.835 and Barlett’s Test of Sphericity was 
significant (X? = 2303.942, DF = 325, p = .000). Determinant was above the 
acceptable 0.0001. The instrument was named the Creativity in Gaming Scale 
(CGS) (see Table 12.1 for factor item loadings and Table 12.2 for variances, means, 
standard deviations and alpha). 

The first factor accounted for the largest variance in the model at 16.3% and 
comprised eight items. Two items were aligned with from the creativity as affective 
change subscale, two items from the learning subscale and four items from the 
transferability subscale. All items related to the general use of creativity or creative 
skills such as problem-solving in other areas of life, except for one item (27.7 I have 
used what I have learnt from games in my job/workplace) which related to the use of 
these skills within the domain-specific context of the workplace. As such, the fac- 
tor was named transferability. The second factor accounted for 13.7% of variance 
and included five items from the creativity as appropriation subscale. As such, the 
factor was named appropriation as all items related to how players could go above 
and beyond what developers expect.The third factor accounted for 11.3% of vari- 
ance and comprised five items with the majority coming from the creativity as 
problem-solving subscale. The remaining two items came from the design affor- 
dances and appropriation subscales. As all items suggested an overall alignment 


Table 12.1 Creativity in Gaming Scale 


Item 


Transferability Appropriation Problem-Solving Affective Design 
change Affordances 
27.7 | have used what | have learnt from games in my 757 
job/workplace 
27.6 Games have influenced my attitudes or behaviours 745 
in other areas of life 
25.1 Playing games has made me come to view things in 729 
everyday life differently 
27.4 Being creative in games gives me a new 722 
perspective on problems and challenges in my 
everyday life 
26.6 Being creative in games has developed my .708 
problem-solving and thinking skills 
25.3 Playing games has made me realise things about .680 
myself 
27.10 | have used the skills and knowledge developed in .657 
games elsewhere in my life 
26.2 | have developed IT/technical skills through being .581 
creative in games 
24.2 | actively seek out glitches .781 
24.5 | enjoy using the game mechanics in new, .779 
unintended ways 
24.3 | try and find shortcuts in games .747 
24.4 | use glitches to enable me to progress in the .743 
game 
(Continued) 
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Table 12.1 (Continued) 


Item Transferability ^ Appropriation 


24.1 | like to test the boundaries of what the game .703 
allows 

24.10 | try and find ways to adapt and bend the rules of .693 
the game 

23.1 | enjoy coming up with new strategies when | play 
games 

24.8 | enjoy creating additional challenges for myself in 
games such as upping the difficulty 

23.5 | highly value the sense of achievement | get when 
overcoming difficult challenges in games 

23.6 | enjoy experimenting with what | can do using 
different game variables 

22.1 | enjoy games that allow me to try out different 
play styles 

25.5 The narrative of a game is important to me 

25.10 | enjoy games which make me question things 

25.6 | enjoy games that give me a new perspective of 
other cultures and societies 

22.7 | prefer games that let me choose the personality 
of my character through dialogue choices 

22.6 Games that allow more freedom for the player are 
more likely to involve creativity 

20.8 Games that allow the freedom to build and make 
things are the most creative 

22.10 Games that allow opportunities to interact with 
the environment help me be more creative in how | 


play 


Problem-Solving 


.802 
.722 
.646 
.641 


.640 


Affective 
change 


.692 
.631 
.627 


.565 


Design 
Affordances 


.789 
.779 
.461 
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Table 12.2 Factor variances, means, and standard deviations 


Factor No.ltems Variance% M SD  Cronbach’s Alpha 
I: Transferability 8 16.3 3.8 83 .86 
2: Appropriation 6 13.7 3.3 75 .86 
3: Problem-solving 5 11.3 4.2 69 76 
4: Affective change 4 8.9 4.3 60 .62 
5: Design affordances 3 7.2 4.0 77 .60 
Whole Scale 26 57.4 3.8 35 .82 


with problem-solving the factor was named problem-solving. The fifth factor com- 
prised four items accounting for 8.996 of total variance. Three of the four items 
came from the creativity as affective change subscale, and one from the design 
affordances subscale relating to dialogue choices (22.7: I prefer games that let me 
choose the personality of my character through dialogue choices). 

Overall, the items aligned with how game narrative can cause players to question 
or reflect on aspects of life, and as such the factor was named affective change. The 
final factor accounted for 7.296 of variance with three items, two of which came 
from the design affordances subscale. The remaining item came from the player 
conceptualisations subscale which was aligned with the view that creativity was 
facilitated by opportunities for object construction (20.8: Games which allow the 
freedom to build and make things are the most creative). Taken together the items pointed 
to an alignment with games that facilitated creativity through environmental inter- 
action and object construction, and as such, the factor was named design affordances. 


12.3.3 Reliability analysis 


Cronbach’s alpha was computed for each subscale individually and the scale as a 
whole. Tiansferability, appropriation and problem-solving produced values of .86, .86 
and .76 which are deemed good (Tavakol & Dennick, 2011). Affective change and 
design affordances had lower values of.62 and.60, respectively; however, these values 
may be attributed to the fact that these two factors had lower numbers of items 
which can reduce the alpha value. Alpha for the scale as a whole was good at.82, 
indicating a high level of internal consistency. 


12.4 Discussion and moving forwards 


The CGS comprised five subscales, three of which aligned to the different 
expressions of creativity; problem-solving, appropriation and affective change, and two 
relating to transferability and design affordances, respectively. Affective change (M = 4.3, 
SD = .60) and problem-solving (M = 4.2, SD = .69) had the highest mean scores, 
suggesting participants had a positive attitude towards games which supported 
these forms of creative expression. Previous work by Cole et al. (2015) found that 
narrative was a core component of the gaming experience which may explain the 
high mean score for affective change.The majority of games include problem-solving 
elements in the form of ill-structured challenges (Kiili, 2005) which players can 
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solve in multiple ways, and due to the ubiquitous nature of these challenges, 
respondents may have been more likely to recognise the statements in the problem- 
solving subscale and hence, score higher on this subscale. Appropriation (M = 3.3,SD 
= .75) on the other hand had a lower mean score, suggesting a more neutral 
valence towards activities associated with creativity as appropriation. As previous 
work found that some players felt glitches detracted them from the gaming experi- 
ence, or were worried about corrupting their saved data (Hall et al., 2020a), two 
statements regarding glitches (24.2, I actively seek out glitches, 24.4: I use glitches to 
enable me to progress in the game) both with lower mean scores (M < 2.5) may have 
impacted on the overall mean score for this subscale. 

Respondents scored fairly high on the design affordances subscale (M = 4, SD = .77), 
suggesting that games which provided greater player freedom, opportunities for 
constructing objects, and supporting interaction with the environment were 
important for cultivating player creativity. The importance of affordances for 
environmental interactivity and object construction is reflected in previous studies 
such as Ward (2015) who argues that options for personalisation of objects and envi- 
ronment provide opportunities for Mini C creativity as players can reflect their 
personal creativity in designing and customising the environment and objects. 
Furthermore, games that provide more freedom for the player such as open-world 
game designs provide a larger "possibility space" (Jarvinen, 2008) which supports 
alternative routes of play,a greater array of strategies and opportunities for exploration. 

Tiansferability (M = 3.8, SD = .83) explained the greatest variance in the model 
(16.396), however, had a less positive mean score. The diversity of items included in 
the subscale could point to the complexity of transferability as a construct, with 
aspects of learning and creativity likely to transfer outside of games. The majority of 
items related to aspects of affective change, lending weight to previous work which 
has suggested games can lead to reflection on experiences (Mekler et al., 2018), new 
perspectives (Oliver & Carr, 2009) and empathy (Bopp et al., 2018; Cole et al., 
2015). A further two items related to specific skill development — namely problem- 
solving and IT/technical skills, which support previous work on the propensity of 
games to develop transferable skills such as problem-solving and IT literacy (e.g., 
Qian & Clark, 2016; Sourmelis et al., 2017; Voulgari et al., 2014). Only one item 
(27.7 I have used what I have learnt from games in my job/workplace) included in the 
scale indicated where creativity and associated skills may be used outside of games; 
however, there 1s the possibility that respondents could have been using domain 
unspecific skills such as problem-solving within a vocational setting. 

Despite adverts being targeted at female technology and gaming groups, the 
sample consisted disproportionately of men (160 males to 82 females). Western 
demographics between male and female gamers is roughly even with 46% of gam- 
ers in the US being female compared with 54% being male (ESA, 2019), with the 
numbers being similar in Europe (ISFE, 2019). As such, future studies may wish to 
try and address this issue by recruiting a more equal sample of both genders and 
gamers from non- Western countries, thus providing a more accurate representation 
of a wider demographic. This would allow the CGS to be further customised to 
accommodate learners throughout the globe. 
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Secondly, the sample was not stratified, and respondents were able to self-select. 
Despite the survey being advertised broadly, there is the possibility that those who 
completed it already self-identified as creative and as a result were more likely to 
take part. As has been argued elsewhere, engaging in gameplay may increase cre- 
ativity and in turn, those who play games may be more creative as a result (Jackson, 
2012; Jackson & Games, 2015; Jackson et al., 2012). As such, future studies may 
wish to compare differences in attitudes between those who self-identify as cre- 
ative and those who do not. Further confirmatory factor analysis (CFA) was not 
performed, and as such, future studies may wish to confirm the factor structure 
using a larger sample. 

Chapter 12 detailed the development of a scale to measure game-specific creativ- 
ity and related constructs. 71 statements on a 5-point Likert scale were subjected to 
PCA analysis, producing a five-factor 26-item solution with sub-scales relating to 
transferability, appropriation, problem-solving, affective change, and design affordances, which 
supported findings from a previous qualitative study. The CGS would not only ben- 
efit game developers in providing guidance on the most important aspects of player 
experience but could also provide benefit for educators and learners. Use of the 
CGS is not simply be confined to classroom-based contexts but could be used in 
other settings such as distance education as an alternative format of learning. 
Moreover, due to their ubiquitous nature, digital games could be used as an alterna- 
tive method of open-world learning where learners have the autonomy to decide 
which games suit their preferences, and educators can use digital games as widely 
available educational tools. In this way, the CGS would allow educators and learners 
to select games that enhance the creative forms of problem-solving, appropriation and 
affective change and assess the development of these over time. 


12.4.1 Implications for practice 


Digital games are enjoyed by millions of people worldwide and offer an alternative 
means for developing creativity — from problem-solving abilities to affective ele- 
ments such as reflection and empathy. For game developers, the CGS provides a 
means of measuring the effectiveness of a given game, in particular the degree to 
which it is facilitating player creativity. A developer could use the scale as a guide 
on what aspects of creativity are most important to players of a given game genre 
or with different previous gaming experiences and demographic characteristics. 
The scale could help learners and teachers capture and reflect on how creativity is 
developing over time and which are those experiences that are more likely to sup- 
port this development. 
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Chapter 13 


Incorporating student opinion 
into opinion mining 


A student-sourced sentiment analysis classifier 


Garron Hillaire, Bart Rienties, Mark Fenton-O'Creevy, 
Zdenek Zdrahal and Dirk Tempelaar 


13.1 Introduction 


Yeah, well, you know, that’s just, like, your opinion, man. 
Jeffrey “the Dude” 
Lebowski talking to Jesus Quintana in The Big Lebowski 


In Open World Learning we focus on free online learning resources and explore 
how to support more students to benefit from these resources. To better under- 
stand the student experience there is a need to focus on emotional measures as 
emotions are considered integral to the learning process (Immordino-Yang & 
Damasio, 2007). As one of the ubiquitous modes of communication in online 
learning is text, we focus on sentiment analysis (SA), which is an affective comput- 
ing measure that can interpret emotions in text by classifying if text is positive, 
negative, neutral, or mixed (both positive and negative). In Chapter 13, we focus on 
how student perceptions relate to and are affected by predictions about their emo- 
tional expression in text. By exploring how student opinions relate to and are 
potentially influenced by SA we explore the validity and utility of SA. 

When SA classifiers are built the process starts with establishing the correct 
labels for text, referred to as ground truth. Establishing ground truth relies on 
human judgements. We ironically reference the quote from The Big Lebowski 
“that’s just, like, your opinion, man.” to light-heartedly call into question how 
truth is established. In our opinion, there is justifiable reason to anchor truth for 
SA to the opinions of students. While there is inherent subjectivity when anchor- 
ing truth to student opinions, SA commonly purports to measure how the opin- 
ion of the author of the text elicits a reaction from the intended reader of the text 
(Balahur & Steinberger, 2009). 

Early SA work used text from product reviews and as well as star ratings (e.g., 
1-star reviews considered negative; 5-star reviews considered positive) (Liu, 2010). 
Effectively, the labels for text were inferred by a star rating that came from the 
author of the text. While it is commonly held that SA technologies work best 
when used on text for similar contexts as to the context where data used to train 
the classifier originated, it is not commonly held that the labels for the text should 
also come from people from the context. For example, a very common practice in 
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SA research is to have researchers rate text using trained raters on established cod- 
ing schemes (Thelwall, 2013), or use anonymous raters from crowd-sourcing plat- 
forms such as Amazon’s Mechanical Turk (MTurk) (Mohammad & Turney, 2013) 
where the wisdom of the crowd typically replaces the training of raters. 

The choice of who is best situated to rate the valence of text is directly related 
to the definition of emotion — which is still a highly debated concept. If emotion 
1s universal, then there are attributes we can identify as characteristics of emotions. 
For example, when someone is happy they might say "I am going to Disneyland". 
This phrase comes from an advertising campaign in the late 1980s where the most 
valuable player from the super bowl would shout this phrase after winning the 
game. While the Disney corporation likely wants this phrase to be universal there 
are people in the world who may have never heard the phrase or even know about 
Disneyland (a popular theme park). In contrast to the universal perspective, the 
Constructed Theory of Emotion (CTE) would suggest that only those with famil- 
iarity of the social context would understand the emotional expression (Feldman 
Barrett, 2018). In Chapter 13, we test CTE by considering the perspectives of the 
social group of students from the classroom and contrast this with a social group of 
anonymous raters. 

In conjunction with the debate on the definition of emotion there is a further 
multi-level debate on how emotion is best measured. The first emotional measure- 
ment debate is between discrete measurement of emotions such as happiness and 
anger in contrast with the perspective that emotion is best measured in dimen- 
sional terms such as the dimension of valence from positive to negative (Feldman 
Barrett & Russell, 1998). In Chapter 13, we focus on the dimensional measure- 
ment of valence. We adopt four possible categories of valence: positive, negative, 
neutral, and mixed. Specifically, we explore if the social consensus used rate text 
should be from a contextual group (the students) or an anonymous out-of-context 
group (Mechanical Turk). Finally, we examine the accuracy of our proposed classi- 
fier by showing the predictions of the classifier to the students during interviews. 
We shared predictions with students to see if students viewed the predictions as 
accurate and useful. To situate this work in the broader context of SA research, we 
first review related work. 


13.2 Related work 


It is important to note that not all emotional measures share a common aim and 
not all measurement adoption explicitly states the assumptions of the measures 
(Weidman, Steckler, & Tracy, 2016).This makes comparison between work difficult 
as SA studies consider accuracy of those measures based on completely different 
definitions of truth (e.g., universal vs. social). Two key assumptions of measurement 
adoption are related to debates both on what emotion is and how it should be 
measured. To illustrate these debates, we review three theories on emotion, three 
approaches to measure emotion in text, and finally classify 15 existing studies in the 
context of learning within this taxonomy of emotional theory and measurement 
based on how they evaluate accuracy of the measures. 
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13.2.1 Three perspectives on emotion 


Basic Emotion Theory (BET) considers some emotional experiences to be so fun- 
damental that they are described as universal. For example, people may have a 
common experience of emotion when it comes to some specific emotional 
responses, such as anger and happiness. Typically, researchers who adopt the BET 
position on emotion focus on five to thirteen emotions that are considered funda- 
mental to the human experience: Happiness, Enjoyment, Sadness, Fear, Anger, 
Disgust, Interest, Contempt, Rage, Love, Lust, Care, and Surprise (Tracy & Randles, 
2011). One limitation for BET is that there is minimal relevance for basic emotions 
in learning activities that span 30 minutes to 2 hours (Calvo & D’Mello, 2010). 

CTE is a perspective that suggests that the manner by which emotion is inter- 
preted is through the influence of social factors. An example of how social theorists 
interpret emotion is illustrated in the book How Emotions Are Made by Lisa Feldman 
Barrett when she used a picture of Serena Williams. The photo was taken immedi- 
ately after Serena beat her sister, Venus Williams, in the 2008 U.S. Open. The pic- 
ture Barrett presents is a cropped image of Serena’s facial expression and Barrett 
suggests that looking at the facial expression in isolation of context might be cat- 
egorised as an expression of terror when using a basic perspective on emotion. 
However, by taking context into consideration we should instead interpret the 
image to mean something closer to exultation (Feldman Barrett, 2018, p. 42). 
Barrett argued that emotion consists of making meaning, prescribing action, regu- 
lating the body, emotion communication, and social influence. Two of the compo- 
nents, emotion communication and social influence, are considered social as they 
are aspects of emotion that cannot be done in isolation. 

Situated Affectivity Theory (SAT) considers the goal as the focal point for inter- 
preting all of the components of emotion (Wilutzky, 2015). With this goal orienta- 
tion, a manipulation between an individual and their environment is the basis for 
stimulation for emotion. The physiological response represents a physical experi- 
ence that resonates with the interaction with the environment. Emotional com- 
munication is thought to be used by people to achieve goals. 


13.2.2 Three perspectives on valence 


Valence is a dimensional perspective on organising emotions commonly consider- 
ing positive and negative. There are three competing perspectives on how valence 
should be organised. The bipolar model considers positive and negative to be the 
opposite ends of the same spectrum (Russell & Carroll, 1999). For example, the 
emotion happy can be placed on the positive end of the spectrum and the emotion 
sad can be placed on the negative end of the spectrum. The bi-variate model sug- 
gests a co-activation where emotions can be categorised as simultaneously activat- 
ing positive and negative (Watson, Wiese, Vaidya, & Tellegen, 1999). In the bi-variate 
model, there are two variables (one for positive and one for negative). The evalua- 
tive space model (ESM) suggests that emotions are both bipolar and bi-variate 
(Cacioppo, Gardner, & Berntson, 1999). Effectively, ESM argues that valence should 
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be thought of as a plane. We can consider the Y-axis of the plane to range from 
neutral to negative and the X-axis of the plane to range from neutral to positive. 
Points on the X- and Y-axes represent bipolar categories of emotion. Chapter 13 
adopts ESM by considering the four valence categories of positive, negative, neu- 
tral, and mixed. 


13.2.3 Reviewing sentiment analysis in education 


SA research shows promise regarding investigations into the complex role of emo- 
tion in learning. Given the potential for SA in educational research, it is essential to 
consider the validity and reliability of SA. To begin considering validity and reli- 
ability it is essential to precisely clarify what SA purports to measure. As it is com- 
mon for researchers to use emotional measures without explicitly stating their 
theoretical perspective on emotion (Weidman et al., 2016), first we reviewed the 
15 identified SA in studies in the context of learning and classified how accuracy 
was evaluated in relation to the three emotion theories reviewed. The results are 
reported in Table 13.1. 

We classified five studies that used methods that are best described as BET. In 
these studies, the researchers believed that they could identify what was accurate as 
this indicated that emotion expression was identifiable by someone other than 
students in the context of learning. For example, BET studies included an exami- 
nation of teacher evaluations where researchers read the teacher evaluations, and 
coded the "actual" sentiments based on the perspective of the researcher reporting 
an overall accuracy of 86.28% (Rajput, Haider, & Ghani, 2016). 


Table 13.1 Interpretation of emotion theory of sentiment analysis studies in 


education 
Studies # BET SAT CTE None 
Ortigosa et al. (2014); Troussas, Virvou, 2 - E — 
Espinosa, Llaguno, & Caro (2013) 
Chaplot et al. (2015); Crossley, Paquette, 4 - - - 


et al. (2016); Wen et al. (2014); Wyner, 

Shaw, Kim, Li, & Kim (2008) 
Calvo & Kim (2010) | - - = 
Munezero et al. (2013); Jagtap & Dhotre 5 - - - 

(2014); Shapiro et al. (2017); Chang, 

Maheswaran, Kim, & Zhu (2013); 

Kagklis, Karatrantou, Tantoula, 

Panagiotakopoulos, & Verykios (2015) 


Rajput et al. (2016); Santos et al. (2013) 2 - - 
Hillaire, Rienties, et al. (2018) l E — 
Total I5 5/15 5/5 3/5 5/15 


Note: BET = Basic Emotion Theory; SAT = Situated Affect Theory; CTE = Constructed 
Theory of Emotion; None = No Evaluation of Accuracy. 
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Three studies used evaluation of accuracy methods best aligned with CTE. For 
example, one study compared course ratings on a Likert scale to determine which 
ratings were positive and inferred comments in the review were positive (Calvo & 
Kim, 2010). None of these studies directly asked participants their opinion about 
the text analysed by SA (sometimes referred to as opinion mining) which is a clear 
gap in educational research. 

We classified five studies all using discussion forums as reflecting SAT when the 
focus was on correlations between SA and outcomes (e.g., student retention), 
because this placed an emphasis on the relationship between emotion expression 
and goal orientation. For example, when predicting student attrition in an online 
course SA was used in conjunction with other measures to generate two predictive 
algorithms which reported a Kappa statistic of 0.403 and 0.432 when predicting 
attrition (Chaplot, Rhim, & Kim, 2015). Next, we evaluated the same 15 studies to 
examine which valence categories were measured considering the four valence 
categories identified in our review on valence theory (see Table 13.2). 

When considering valence categories measured when applying SA to the con- 
text of learning, there appears to be an emphasis in the existing literature on mea- 
suring positive and negative valence. Of the 15 studies reviewed, all of the studies 
measured both positive and negative valence as indicated in Table 13.2. About half 
of the studies, seven out of 15, measured the category of neutral, and only two out 


Table 13.2 Valence categories of sentiment analysis studies in education 


Studies # Positive Negative Neutral Mixed 


Munezero, Mozgovoy, Montero, 7 — — 
& Sutinen (2013); Jagtap & 
Dhotre (2014); Troussas, 
Virvou, Espinosa, Llaguno, & 
Caro (2013); Crossley, 
Paquette, Dascalu, 
McNamara, & Baker (2016); 
Wen, Yang, & Rosé (2014); 
Wyner, Shaw, Kim, Li, & Kim 
(2008); Chang, Maheswaran, 
Kim, & Zhu (2013) 
Calvo & Kim (2010); Ortigosa — 6 = 
et al. (2014); Chaplot et al. 
(2015); Hillaire, Rienties, 
et al. (2018); Shapiro et al. 
(2017); Kagklis, Karatrantou, 
Tantoula, Panagiotakopoulos, 
& Verykios (2015) 
Rajput et al. (2016) | - 
Santos et al. (2013) | 
Total 15 15/15 15/15 7/15 2/15 


KIK 
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of 15 studies measured a category of mixed emotion. One study (Santos et al., 
2013) measured all four categories of positive, negative, neutral, and mixed. 
However, they referred to mixed as ambivalence - which they defined as both 
positive and negative. In the second study considered mixed expression (Rajput 
et al., 2016), the authors used neutral and mixed interchangeably when describing 
the results but reported statistics for the category of mixed expression. 


13.3 Student-sourcing, crowd-sourcing ground truth for 
a classifier with students 


We explore centring students with student sourcing, using crowd-sourcing methods 
with students evaluating their own group discussions. In doing so we flip the 
assumption from the perspective that crowd ratings are by default noise to the 
default assumption they are accurate. When establishing ground truth more single 
labels are better in the condition where raters are considered reliable. Based on the 
shifting the assumption that student ratings are by default accurate single ratings is 
considered useful. A common approach is using the Expectation Maximisation 
(EM) algorithm (Dempster, Laird, & Rubin, 1977), which selects the best label 
using crowd-sourcing label data by considering both the prevalence of each 
valence category and the categorical accuracy of each rater. Effectively the unique- 
ness of student opinions is favoured by this approach because the EM algorithm 
adopts single ratings as ground truth. Where multiple ratings occur, the EM algo- 
rithm selects a best fit as a proxy for what social consensus might evolve between 
students. As the approach is novel we evaluate the work using both standard 
approaches to reliability, and benchmark this specialised classifier with general 
crowd-sourcing approaches. 

Typically, with crowd sourcing a large number of people are recruited to cate- 
gorise text by providing labels frequently generating five labels for each item 
being categorised. Providing both Fleiss’ Kappa and Krippendorff 5 alpha are sug- 
gested for crowd-sourced labels in social computing (Salminen, Al-Merekhi, Dey, 
& Jansen, 2018) because the expectation is that agreement is usually low with 
crowd-sourcing methods. For example, Krippendorf’s alpha scores around 0.10 
were frequently found when evaluating crowd-sourcing methods (Alonso, 
Marshall, & Najork, 2013). We use crowd sourcing as one of the benchmarks for 
student's sourcing where students provide labels instead of anonymous M' Turk 
raters disconnected from the classroom context and then validated the outcome 
of training on MTurk ratings by predicting student labels that close the loop by 
validating with student labels. This approach is contrasted with centring students 
where using artificial intelligence approaches we instead train a classifier based on 
student labels and then use the student-sourced classifier to predict student labels 
(see Figure 13.1) 

Finally, we conducted interviews with respective students involved in the exper- 
iments to further lean into student perspectives. Therefore, to investigate the 
assumption that we can accept student opinions as correct for opinion mining two 
research questions need to be assessed: 
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Train/Test Validation 


Ground Truth . 
Come D. (a) Closing the Loop 


Centring Students 


Figure 13.1 Comparing closing the loop with centring students with ground 
truth. 


1 To what extent are student-sourced examples reliable? 
2 To what extent can we use student-sourced examples to train an SA classifier? 


13.4 Methods 


Chapter 13 is based on two extensive studies undertaken with two separate cohorts 
of students at a university in the Netherlands as part of the thesis of the first author 
(Hillaire, 2021). While the university recruits international students, courses are 
taught in English. Cohort 1 included 767 freshmen in a statistics course in Fall 
2016 who (1) worked on an online group assignment where students chat with 
one another and (2) reviewed their discussions and provided examples of messages 
for valence categories. There were 304 females and 463 males. The population was 
international, including 191 domestic (Dutch), 529 European Students, and 47 
non-European students. Mechanical Turk was used to generate five labels for mes- 
sages selected by Cohort 1 Students. Cohort 2 included 484 freshmen in the same 
statistics course in Fall 2017 who (1) completed an online group assignment, (2) 
provided examples of messages for valence categories (see Figure 13.2). 

We generated Data Set 1 with the EM algorithm which selected the ground 
truth label for each text message based on the example text and labels provided by 
Cohort 1 Students. We generated Data Set 2 with the EM algorithm which selected 
the ground truth label for text message from text examples provided by Cohort 1 
Students and labels for the text provided by Mechanical Turk workers. We gener- 
ated Data Set 3 with the EM algorithm which selected the ground truth label for 
each text messages based on the example text and labels provided by Cohort 2 
Students (see Figure 13.2). 

Finally, we used Data Set 1 to train Classifier 1 (a logistic regression classifier). 
We used Data Set 2 to train Classifier 2 (a logistic regression classifier). Both 
Classifier 1 and Classifier 2 categorised text messages as positive, negative, neutral, 
or mixed. Finally, we used Classifier 1 and Classifier 2 to predict labels for Data 
Set 3. To ground the comparison between Classifier 1 and Classifier 2 we 
compared them to general SA classifiers used on Data Set 3 (see Figure 13.2). 
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Cohort 1 Students EM Algorithm Mechanical Turk EM Algorithm 
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Cohort 2 Students EM Algorithm 


Figure 13.2 Research design generating three data sets and two classifiers. 


Finally, we interviewed six students from Cohort 2 to evaluate the accuracy of 
Classifier 1 as well as the utility of having access to SA predictions on their own 
text messages. 


13.5 Procedure 


In Cohort 1, students (n = 767) were assigned randomly to groups of five (M = 
4.73 SD = 0.84) in a laboratory setting, whereby each student had a desktop com- 
puter, and all written communication was online as part of a regularly occurring 
lab session for their course. Previous research on this task reported that overall 
students enjoyed working together in groups (Mittelmeier, Rienties, Tempelaar, 
Hillaire, & Whitelock, 2018). The group work activity for Cohort 2 was the same 
as for Cohort 1 with a small change to the warmup exercise. The post-activity was 
changed in that participants no longer provided examples of ambiguous messages 
and the final modification was a series of interviews conducted with six students 
to examine the trustworthiness of the algorithm’s predictions. 

In the post-activity Cohort 1 participants were first given a set of instructions to 
provide 1-3 examples of positive, negative, neutral, mixed, and ambiguous messages 
(for Cohort 2 removed the Ambiguous valence category). For Cohort 2, the inter- 
view consisted of three parts. Part 1 asked students to review a subset of messages 
from their group chat and identify if the message was positive, negative, neutral, or 
mixed. Part 2 asked participants to compare their rating with the prediction from 
the student-sourced classifier in conjunction with the text features the algorithm 
used to predict the valence. If the prediction was different than the student label 
provided in Part 1 the student was asked if the algorithm prediction changed their 
mind. Finally, at the end of the interview participants were asked if the predictions 
were useful. 
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13.5.1 Analysis 


To answer RQ1 we computed inter-rater agreement for Data Sets 1, 2, and 3 
and compared the results to benchmarks of agreement for crowd sourcing in 
social science. Low agreement in crowd ratings does not mean the opinions of 
labels are incorrect; it may simply indicate they have different opinions (Salminen 
et al., 2018). 

To answer R Q2 we generated Classifiers 1 and 2 (logistic regression classifiers) 
based on Data Sets 1 and 2, respectively, and compared the accuracy of Classifiers 1 
and 2, with General Benchmarks when predicting valence labels for Data Set 3. We 
also benchmarked the accuracy of Classifiers 1 and 2 with general measures. Finally, 
we interviewed students from Cohort 2 to evaluate the accuracy and utility of 
predictions from Classifier 1 used to interpret their text data. 


13.5.2 Results 


To answer RQ1, we first established three datasets and then computed agreement 
statistics. Data Set 1 was generated by 767 students providing examples for positive, 
negative, neutral, mixed and ambiguous, resulting in 2512 records with 1979 dis- 
tinct messages. Data Set 2 was generated by using the EM algorithm to select the 
ground truth label for Data Set 1 which resulted in 1778 messages categorised as 
positive, negative, neutral, and mixed (we excluded the 201 messages categorised as 
ambiguous). We next used Mechanical Turk where five raters classified the 1778 
messages as positive, negative, neutral, and mixed. Data Set 3 was generated by 484 
students providing examples for positive, negative, neutral, mixed. This resulted in 
986 records with 755 distinct messages. After generating the three datasets we 
computed agreement statistics resulting in Krippendorff's alpha scores of 0.44, 
0.25, and 0.42 for Data Sets 1, 2, and 3, respectively. Datasets 1 and 2 generated a 
range of between one and five ratings per unique message so we further computed 
and report Fleiss' Kappa scores for agreement statistics based on the number of rat- 
ings. For Data set 2 we had five raters for every unique message and report Fleiss’ 
Kappa for completeness (see Table 13.3). 

In Table 13.3, we observe that both Data Set 1 (alpha=0.44) and Data Set 3 
(alpha=0.42) had similar Krippendorff’s alpha scores indicating (1) students had 
moderate agreement with one another on the valence labels from their own chat 
data; and (2) student agreement was above Mechanical Turk raters (alpha=0.25) 
as well as below the average Krippendorff 5 alpha score of 0.60 found in crowd- 
sourcing studies in social science. These results show promise that crowd sourcing 
with students has the potential to do better than using services such as Mechanical 
Turk, but also indicates that agreement is below the average indicating room for 
improvement. 

To answer RQ2, we first established a series of benchmarks using general SA 
technologies making predictions about Data Set 3 with labels from Cohort 2 stu- 
dents. The f-measures for the best benchmarks was VADER. with an f-score of 
0.43. Next, we trained Classifier 1 using Data Set 1 with labels from Cohort 1 


Table 13.3 Agreement statistics for three data sets 


Raters l 2 3 4 5 Krippendorff’s alpha 
rater raters raters raters raters 

Data Set | Cohort | 1586 330 56 6 | 0.44 
(Fleiss’ Students (-) (0.42) (0.52) (0.30) (-) 

Kappa) 
Data Set 2 Mechanical Turk 1778 (0.25) 0.25 
(Fleiss’ 

Kappa) 
Data Set 3 Cohort 2 577 139 (0.41) 30 (0.50) 4 5 0.42 
(Fleiss’ Students (-) (0.36) (-0.15) 


Kappa) 


|t 39 eJre||iH UOLUeD 081 
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students and trained Classifier 2 using Data Set 2. The best cross-validation f-scores 
for Classifier 1 was 0.475 and the subsequent validation F-score was 0.462. The 
best cross-validation f-scores for Classifier 2 was 0.550 and the subsequent valida- 
tion F-score is 0.456. When comparing these results Classifier 2 had a higher cross- 
validation score and Classifier 1 had a higher validation score (see Table 13.4). This 
means that when we tested the two classifiers on Data Set 3 (with labels from 
Cohort 2 students) that Classifier 1 trained on Cohort 1 student labels was more 
accurate than Classifier 2 trained on Mechanical Turk labels. 

Finally, we interviewed six students from Cohort 2 about the predictions from 
classifier 1. Across the six students interviewed they reviewed 113 messages of 
which they agreed with the algorithm 36 times, and disagreed 77 times. For the 77 
disagreements, they changed their mind to agree 21 times (27% or 21/77) after 
seeing the algorithm's predictions (see Table 13.5). When considering the initial 
agreement (36 times) and when they changed their mind (21 times) the students 
considered the prediction accurate 50% of the time (57/113). 

Participants changed their mind to agree with the algorithm one to three times 
with the exception of one student who changed their mind eleven times. Students 
who found the algorithm to be useful had final agreement that ranged from 42% 
to 67% (initial agreements 5-9 messages; final agreements 8-20 messages) with a). 
The one student who did not find the algorithm to be useful, Student-6, only 
initially agreed with the algorithm once and changed their mind to agree with it 
two times for a total of three agreements out of 12 messages (25%). While sample 


Table 13.4 Agreement statistics for three data sets 


Train/test data Validation data Cross validation Validation 
Classifier | Data Set | Data Set 3 0.475 0.462 
(Cohort | (Cohort 2 
Students) Students) 
Classifier 2 Data Set 2 Data Set 3 0.550 0.456 
(Mechanical (Cohort 2 
Turk) Students) 


Table 13.5 Agreement, disagreement, final agreement, and usefulness of SSAC 


Participant Agree Disagree (change) Final agreement % Useful 
Student 7 10 (1) 47% Yes 
Student 2 9 21 (11) 67% Yes 
Student 3 8 9 (2) 59% Yes 
Student 4 5 13 (3) 44% Yes 
Student 5 6 13 (2) 42% Yes 
Student 6 I 11 (2) 25% No 
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size from interviews is small, it is noteworthy that Student-6 who did not find it 
useful had a final agreement of 25%, which was the same as the unweighted chance 
levels of accuracy for predicting four categories, while all of the students who 
found it useful had above chance levels of agreement. This result suggests above 
chance levels of accuracy is necessary for students to find the classifier useful. Five 
out of six students interviewed said that the algorithm was useful. When describing 
the usefulness of the algorithm, participants described benefits including: (1) better 
understanding their own communication (e.g., “I started thinking more about 
what I said”), (2) better understanding communication of other students (e.g. “I 
started analysing the way others said it”), and (3) seeing an alternate interpretation 
that changed their mind which they described as learning from the algorithm. 


13.6 Discussion and moving forwards 


Chapter 13 illustrated how a student-sourced SA could build a better understand- 
ing of the online student experience and emotions in particular. What is novel 
about our findings is that we demonstrated that (1) student labels had a higher 
level of inter-rater agreement than Mechanical Turk labels, (2) Mechanical Turk 
labels generated a higher cross-validation score than student labels, and (3) student 
labels trained a classifier with higher accuracy than the classifier trained using 
Mechanical Turk labels. A potential explanation for this result is that the consensus 
established by Mechanical Turk workers was simply divergent from the consensus 
of students. We could reframe this to say what Mechanical Turk workers consider 
to be the true labels for text has higher consistency, but their idea of truth is dif- 
ferent from students. From the perspective of the CTE the consensus established 
by members of the social context is the very definition of emotion. Interpreting 
these results from a CTE perspective suggests there is potential benefit in having 
raters that come from the context where the text was originally generated. This 
finding builds on the existing belief that SA classifiers are context sensitive and 
perform best when used in contexts similar to the context where training data for 
the classifier was collected by contributing evidence that context sensitivity may 
also include the relationship between the raters of text and the context where the 
text was collected. 


13.6.1 Implications for practice 


Practitioners that use educational technology should be cautious when they incor- 
porate SA classifiers trained on data dissimilar to classroom data as general tech- 
nologies had low performance. Practitioners should consider how to centre the 
lived experience of students when integrating classifiers that seek to model highly 
subjective topics such as SA. Not only is there reason to share SA predictions with 
students to anchor accuracy with student opinion, but students reflecting on SA 
predictions demonstrated the benefit of thinking about both what they say to their 
peers and what their peers say to them in terms of emotional expression. Future 
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work should explore supporting and evaluating student awareness of emotion 
expression in text. 
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Informing learning design in online 
education using learning analytics 
of student engagement 


Quan Nguyen, Bart Rienties and Denise Whitelock 


14.1 Introduction 


Since the early 2000s, two strands of research in education have emerged that can 
help educators gain better insights into the teaching and learning process. These are 
learning design (LD) and learning analytics (LA). Learning design, in this context, 
is defined as “a descriptive framework for teaching and learning activities (“educa- 
tional notation”), and to explore how this framework can assist educators to share 
and adopt great teaching ideas.” (Dalziel et al., 2016, p. 4). Research in LD has 
developed a wide range of tools and frameworks to document and visualise 
sequences of learning activities designed by teachers and to guide them through 
the LD process (Cross et al., 2012; Laurillard et al., 2018). Through the transition 
from implicit to explicit representations of LD, teachers can reflect on their prac- 
tices, while re-using and adapting good instructional approaches from others 
(Agostinho et al., 2011). 

In parallel to LD, LA has emerged as a field in the decade since the first Learning 
Analytics Knowledge (LAK) conference in 2011. Learning analytics is defined as 
“the measurement, collection, analysis and reporting of data about students and 
their contexts, for purposes of understanding and optimising learning and the 
environments in which it occurs” (Ferguson, 2012, p. 305). LA research typically 
collects a large amount of data about students such as demographics, course per- 
formance, activity logs of students (Rienties et al., 2019; Tempelaar et al., 2018), 
discussion forums interactions (Wise et al., 2017), and open texts from essays or 
course evaluations (Ullmann, 2019). By taking advantage of advanced analytical 
techniques such as machine learning (Ullmann, 2019), text-mining (Hillaire et al., 
2022), and social network analysis (Wise et al., 2017), LA has created practical 
applications to support the learning process. 

There exists a strong synergy between the two fields (Lockyer & Dawson, 2011). 
On the one hand, LA provides data and tools to test pedagogical assumptions in 
LD against actual student interactions. On the other hand, LD provides the neces- 
sary contextual overlay to better understand observed student behaviour and trans- 
late LA findings into actionable insights (Lockyer & Dawson, 2011). Prior empirical 
works have shown the benefits of embedding LD in LA such as improving predic- 
tive accuracy of academic performance (Gašević et al., 2016), understanding the 
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impact of LD on student engagement, satisfaction, and performance (Rienties & 
Toetenel, 2016), exploring the variety of designing approaches (Nguyen et al., 
20172, 2017b, 2017), uncovering the (in)consistency between students’ engage- 
ment and instructors’ learning design (Nguyen et al., 2018a, 2018b, 2018), and 
highlighting in Chapter 8 that geocultural background characteristics of learners 
influence engagement with LD (Rizvi et al., 2022). The next sections will outline 
some applications of learning analytics to inform learning design in online educa- 
tion, using large-scale empirical findings from over three years of extensive research 
at the Open University UK (OU). 


14.2 Background 
14.2.1 Study context 


All the studies reported in this chapter took place at the OU. The OU is the largest 
academic institution in the UK and in Europe with over 170,000 enrolled students. 
As a pioneer in distance learning model since 1969, the OU offers more than 200 
qualifications and 400 modules via a distance learning model, which involves the 
use of a Virtual Learning Environment (VLE) in conjunction with online and/or 
face-to-face tutorials with designated tutors. The OU is a leading research institu- 
tion in learning analytics (Clow, 2013; Ferguson, 2012; Herodotou et al., 2019). For 
example, OU Analyse produces an early prediction of “at-risk” students based on 
their demographic data and their interaction with the VLE (Kuzilek et al., 2015). 
In addition, the OU has a rich database of its students including demographics, 
academic performance, course registration, and trace data of activities on VLE 
across hundreds of thousands of students since 1970s. Therefore, the OU provides 
a unique opportunity for LA researchers to address educational research questions 
at a large-scale, improving the external validity and generalisability of the 
findings. 


14.2.2 Learning design at the Open University 


Compared to other universities, the OU module production process is longer and 
more complex. This process typically takes two to four years, involving multiple 
stakeholders with specialised skills in academic content writing, teaching, project 
management, media production and technical development (Cross et al., 2012). 
Capturing and quantifying pedagogical practices is challenging, to say the least. At 
the OU, each new module goes through a mapping process, which maps out all 
learning activities and their estimated time to complete the activities. Learning 
activities are categorised based on the learning activity taxonomy originally devel- 
oped by Conole et al. (2008), which has subsequently been further fine-tuned and 
adjusted over time based upon both practical experiences as well as LD research 
(Toetenel & Rienties, 2016a, 2016b) (Table 14.1). 

Assimilative activities refer to tasks which require student’s attention to informa- 
tion. These include watching lecture video, reading the text, listening to an audio 
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Table 14.1 Learning activity taxonomy 


Taxonomy 


Type of activity 


Example 


Assimilative 


Finding and handling 
information 


Communication 


Productive 


Experiential 


Interactive/adaptive 


Assessment 


Attending to information 


Searching for and 
processing information 


Discussing module related 
content with at least one 
other person (student or 
tutor) 

Actively constructing an 
artefact 


Applying learning in a 
real-world setting 


Applying learning in a 
simulated setting 


All forms of assessment 
(summarive, formative 


Read, Watch, Listen, 
Think about, Access. 

List, Analyse, Collate, 
Plot, Find, Discover, 
Access, Use, Gather. 

Communicate, Debate, 
Discuss, Argue, Share, 
Report, Collaborate, 
Present, Describe. 

Create, Build, Make, 
Design, Construct, 
Contribute, Complete. 

Practice, Apply, Mimic, 
Experience, Explore, 
Investigate. 

Explore, Experiment, 
Trial, Improve, Model, 
Simulate. 

Write, Present, Report, 
Demonstrate, Critique. 


and self assessment) 


Source: Retrieved from Conole et al. (2008). 


file, etc. Finding and handling information activities implies, for example, searching 
and filtering for relevant literature in a particular topic on the internet. Communication 
activities refer to a range of practices to communicate such as posting in a discus- 
sion forum and replying to peer comments. Productive activities represent the con- 
struction of an artefact, such as writing a summary or resolving a problem. 
Experiential activities provide students with opportunities to apply theories in a 
real-world setting such as case study, or field trip. Interactive/adaptive activities 
encourage students to apply what they learned in an experiential environment or 
interacting with a simulation. Finally, assessment activities evaluate the student's 
understanding such as writing through the construction of an essay, exam or mak- 
ing a presentation (Conole, 2012; Conole et al., 2008). 

For each learning activity, an estimation is made for how long it would take an 
average student to complete that activity. This estimation is usually determined by 
the module team and being embedded in the module guide on the VLE as a guid- 
ance for students’ study time allocation (Figure 14.1). If the time estimation is not 
explicitly stated in the module guide, it will be determined using agreed conven- 
tions for study speed and amount of time allocated to studying figures, tables, 
images, audio and video within module materials. The time estimation of each 
learning activity was aggregated at a weekly level (i.e., estimated workload per 
week). The workload of each module was restricted by its number of credits, with 
each credit equates one hour of studying. For example, a 30-credit module requires 
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Activity 3.3 


( Spend approximately 35 minutes on this activity. 


The Open University provides guidance on a number of study skills on the 
StudentHome website. You are encouraged to refer to these guidance notes and 
excercies as and when it seems useful to you. However, from time to time you will be 
explicitly directed to one of these OU study skills sessions as part of one of the activities 
on B100. This is because much of the generic guidance available in the sessions is very 
relevant to studying B100 and it would be an unnecesary doubling of effort to recreate 
very similar guidance here, specificlly for B100. Also, by linking to the generic study 
skills pages you will get an idea of what else is available there that may be useful to you 
at some point. This activity on note taking is one of those where you are encouraged to 
work through a generic study skills session before you return to the B100 website to 
complete the activity with some B100 specific material. 


Figure 14.1 Time estimation of learning activity in a module guide. 


ESV MEEUW ->i Learning Outcomes] aia Activity Planner | Æ Workload Too! pam Module Map] && Design Log Evaluation 


Hours spent undertaking each type of activity 


Design stages 


Finding and Total hours 


Week Assimilative handling Communication Productive Experiential debtor" Assessment sip 228 ua 

information 6-76, SD ie 

+ Week 1 5.24 ^| 01 ^| 0 ^| 0 ^| 0 4| 0 4| 01 4| 544 i 
+) Week 2 4.59 z| o5 z| o ^| 0.95 ^| o z| 0 z| 0 7| 604 
+| Week 3 3.22 7) 136 ^| 0 7) 0.55 Z| o3 2| 0.73 ^| os 7) 646 
+) Week4 5.93 ^| 0 ^| 0 4| 0.35 4| 0 4| 0 4| 0 4| 6.28 


Figure 14.2 Activity planner tool. 


300 hours of learning or 8-9 hours per week, and a 60-credit module requires 600 
hours of learning or 16-18 hours per week. 

When using data to compare module design across disciplines and modules, 
according to previous work (Rienties & Toetenel, 2016; Toetenel & Rienties, 
20162) it 1s important to classify learning activities in an objective and consistent 
manner. In particular, each module goes through a mapping process by a module 
team which consists of an LD specialist, an LD manager, and faculty members. This 
process typically takes between one to three days for a single module, depending 
on the number of credits, structure, and quantity of learning resources. First, the 
learning outcomes specified by the module team were captured by an LD special- 
ist. Each learning activity within the module's weeks, topics, or blocks was catego- 
rised under the LD taxonomy and stored in an "activity planner" — a planning and 
design tool supporting the development, analysis, and sharing of LD (Figure 14.2). 
Next, the LD team manager reviews the resulting module map before the findings 
are forwarded to the faculty. This provides academics with an opportunity to com- 
ment on the data before the status of the LD 1s finalised. To sum up, the mapping 
process is reviewed by at least three people to ensure the reliability and robustness 
of the data relating to LD. 
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14.3 How instructors design online courses 


To understand how instructors designed online courses at the OU, a subset of 37 
modules were extracted from the OU Activity Profile tool, which was mapped on 
a weekly basis. These modules were selected based on the data availability in con- 
junction with discussion with the LD team, in order to determine a representative 
sample of the OU courses. Table 14.2 provides descriptive information about the 
selected modules. 

In line with previous findings (Rienties & Toetenel, 2016; Toetenel & Rienties, 
20162), assimilative, assessment, and productive activities were the predominant 
types of learning activity (Table 14.3). Assimilative activities accounted for half of 
the workload on average (M = 50.0096, SD = 13.03%), followed by assessment 
(M = 24.496, SD = 8.38%) and productive (M = 17.6096, SD = 12.39%). There 
was a large variation in terms of the total workload across modules. All modules 
have some proportions of assimilative, productive, and assessment but some mod- 
ules did not have any communication, finding information, interactive, or expe- 
riential activities (Table 14.3). 

A visual comparison of LDs across the four disciplines suggested that STEM 
modules were more likely to use experiential and interactive activities than other 
disciplines (Figure 14.3). Modules in Education, Health, and Languages had the 
highest proportion of workload for productive activities. A Kruskal-Wallis test indi- 
cated that the differences between disciplines in productive (X? = 14.37, p < .01) 
and experiential activities (X? = 8.64, p « .05) were statistically significant. These 
results based on aggregated figures of LD confirmed findings from previous studies 
(Rienties & Toetenel, 2016;Toetenel & Rienties, 20162). It also added new insights 


Table 14.2 Descriptive statistics of 37 modules 


Frequency Percent 


Credits 
30 17 45.9% 
60 20 54.1% 

Level 
0 3 8.1% 
l 26 70.3% 
2 5 13.5% 
3 3 8.1% 

Faculty 
Arts & Social Sciences 9 24.3% 
Business & Law 6 16.2% 
Education, Health, 

Languages 9 24.3% 

STEM 13 35.1% 


Note: Level |, 2, 3 at the OU are equivalent to introductory, 
intermediate, and advanced courses. Level 0 represents access 
modules. 
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Table 14.3 Descriptive statistics of seven types of learning activity in 37 modules 


N Minimum | Maximum Mean Std. deviation 
Assimilative 37 23% 75% 50.0% 13.03% 
Information 37 0% 8% 2.2% 1.98% 
Communication 37 0% 9% 2.5% 2.96% 
Productive 37 2% 59% 17.6% 12.39% 
Experiential 37 0% 12% 1.1% 2.25% 
Interactive 37 0% 19% 2.2% 4.56% 
Assessment 37 13% 57% 24.4% 8.38% 


Metric = % of total workload. 


Discipline 
E Arts & Soc 
Business & Law 
Education, Health & Languages 


60% 7 | STEM 


40% | " 

20% | T? 
TON i 

096 T pim ra. ma Lh I aon mi m BH T 


Assimilative Information Communication Productive Experiential Interactive Assessment 


Errors Bars: +/- SE 


Percentage of total workload 


Figure 14.3 A comparison of seven learning activity types across four disciplines. 


into the disciplinary differences in LDs. The next section will unpack the changes 
in LDs over time. 

Figure 14.4 visualised the changes in total workload of 37 modules over 31 weeks 
grouped by the number of credits. By default, the total workload of 30 credit mod- 
ules was lower than 60 credit modules. However, there were a lot of fluctuations in 
workload across modules over time (Ma, 4,4; = 6.5, SD 39 creait = 3.11; Moo creat = 8.9, 
SD, credit = 4-42) with a slight decrease in the last 4 weeks toward the end of the 
module (Table 14.4). 

Figure 14.5 illustrates the average time students were expected to spend per 
module (in hours) on different learning activities over 34 weeks. Each colour 
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Figure 14.4 Visualisation of total workload over time of 37 modules over 3l 
weeks. 


Table 14.4 Descriptive statistics of seven learning activity types of 37 modules 
over 3| weeks 


N Min Max Mean Std. 
deviation 
30 credits modules 
Assimilative 475 0.0 12.4 3.1 2.36 
Information 475 0.0 2.3 0.1 0.31 
Communication 475 0.0 2.5 0.2 0.38 
Productive 475 0.0 9.5 1.3 1.39 
Experiential 475 0.0 9.0 0.1 0.71 
Interactive 475 0.0 4.4 0.2 0.78 
Assessment 475 0.0 10.5 1.3 2.23 
Total 475 0.0 23.6 6.4 3.11 
60 credits modules 
Assimilative 613 0.0 15.0 4.5 3.88 
Information 613 0.0 13.0 0.3 0.92 
Communication 613 0.0 11.0 0.3 0.89 
Productive 613 0.0 12.5 1.3 1.94 
Experiential 613 0.0 1.8 0.0 0.17 
Interactive 613 0.0 19.1 0.1 0.87 
Assessment 613 0.0 20.0 2.4 4.06 
Total 613 0.0 35.9 8.9 4.42 


Metrics = Hours N = Number of data points per module per week. For example, a 30- 
week long module has 30 data points. 


196 Quan Nguyen et al. 


Hours 
A 


et Pea? 


o eara neis alec tae Teen i Eine ET TE NAE Enna. 
123 45 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 
Study week 
li Assessment -- Communiction 
E Assimiliatve Experiential 
Productive ++» Information 
© Total Interactive 


Figure 14.5 Visualisation of seven learning activity types over time of 37 modules 
over 3| weeks. 


represents a type of learning activities. The visualisation also confirmed the domi- 
nance of assimilative (orange), assessment (blue), and productive (purple) learning 
activity types. Assimilative activities were present throughout most of the learning 
process except for the last four weeks and accounted for half of the total workload 
(M = 3.90, SD = 3.37). 

Interestingly, there was an opposite trend between assimilative and assessment 
activities throughout the course (r — -0.462, p « .01). More assimilative activities 
were used at the beginning of a module, whereas more assessments were used 
toward the end. Assessment activities were also negatively correlated with other 
types of learning activity (Table 14.5). In other words, teachers deliberately reduced 
the workload of other learning activity types when an assessment was activated. 

After capturing a dynamic picture of LD over time, we took a further step to 
investigate how LDs were configured across different modules. We reported four 
exemplary modules across four disciplines with a variety of configurations and pat- 
terns of learning activities (Figure 14.6). In line with the findings above, all four 
modules extensively made use of assimilative (orange), productive (purple), and 
assessment activities (blue). However, there are subtle differences in the way each 
module utilised these three activity types. 

The first module in Arts followed a "traditional" design, with a lot of reading, 
watching, listening activities. Its assessment consisted of five continuous assess- 
ments, so-called Tutor Marked Assessments (TMAs) every 4—5 weeks and an end 


Table 14.5 Correlation analysis of seven learning activity types over time of 37 modules 


| 2 3 4 5 6 7 
|. Assimilative l 
2. Information .082* | 
3. Communication .166* .167* | 
4. Productive 161% .167* .130* l 
5. Experiential .021 -.021 -.022 -.002 | 
6. Interactive .016 .015 .050 .008 .012 | 
7. Assessment -.462* -.115™ -.124* -.292* -.062* -.003 l 
8. Total .555* .248" .300™ .362"* .078* .230™ .283" 


** Correlation is significant at the 0.01 level (2-tailed). 
* Correlation is significant at the 0.05 level (2-tailed). 


N = 1088 data points. 


Note: Adapted from Nguyen et al. (2017). 
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Figure 14.6 Four exemplary modules from Arts, Health, Science, and Languages. 


of module assessment (EMA) in week 30. The workload of this module was rela- 
tively constant for most parts, except for the peak in week 8 which had a double 
amount of workload (i.e., 11.31 hours) compared to other weeks. 

The second module in Health was similar to the first module in Arts. However, 
the former used more productive activities and had two-week long studying time 
for each TMA.The third module in Science adopted a continuous assessment strat- 
egy using a lot of quizzes throughout the learning process. This module also had a 
considerable amount of interactive (yellow) and experiential (light blue) activities 
compared to other modules. There were several dips in workload in week 14, 19, 
and 25 which represented TMA preparation weeks. The dip in week 29 repre- 
sented an EMA preparation week. The fourth module in languages had a higher 
mix of assimilative and productive activities. There was also a higher presence of 
communication activities (red) in this module. 

In summary, the findings have started to demonstrate through visualisations and 
statistical analysis the overall trends in LDs across 37 modules over 31 weeks. The 
three main types of learning activity namely assimilative, productive, and assessment 
were visible in all modules. There was a moderate negative correlation between 
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assimilative and assessment activities. A closer look into each module individually 
revealed subtle differences in how each teacher utilised each learning activity type. 


14.4 How students engage in online learning activities 


To capture student engagement, the time spent on the VLE (1.e., Moodle) was used 
as a proxy of student online behavioural engagement. Time spent on the VLE was 
calculated as the duration between two consecutive clicks, aggregated across all 
study sections. The data were captured from four weeks before the start of the 
module until four weeks after the end of the module. Learning activities were 
planned for over 30 weeks. In order to link LD data with engagement data, the 
measurements needed to be on the same level of analysis (e.g., weekly). Based on 
this, behavioural engagement were generated as the average time spent on the VLE 
per week (in minutes). 

Fixed effect models were carried out with the average time spent on VLE per 
week (Table 14.6) as a dependent variable. For each predictor, four models were 
applied. First, we ran a normal OLS regression model. Second, a fixed-effect model 
was used to control for the unobserved heterogeneity of time. Third, we controlled 
for the fixed effect between modules. Finally, we controlled for the fixed effects of 
both time and modules. Since assimilative activities account for most of the 


Table 14.6 Fixed effect model of VLE engagement per week predicted by learning 
design activities 


DV = VLE per week Unstandardised coefficients 
(1) (2) (3) (4) 
MODELS OLS FE_week FE_module FE_module_week 
Assessment 4.98** 5.09** 2.47** 2.80** 
(0.57) (0.59) (0.47) (0.47) 
Information 2.40 3.23 -0.72 0.15 
2.64 2.60) 1.98 1.94 
Communication 26.29** 26.29** 16.54*** 17.44** 
(2.66) (2.62) (2.16) (2.11) 
Productive 1.75 1.73 -1.84 -1.83 
(1.14) (1.12) (1.04) (1.03) 
Experiential 3.57 4.49 -2.07 -0.99 
(3.83) (3.78) (2.98) (2.91) 
Interactive 11.57*** | 1.25** -0.33 -0.46 
(2.23) (2.20) (1.81) (1.78) 
Constant 95.66** 95.30** 110.6** 172.|** 
(2.91) (2.85) (2.46) (10.50) 
Observations 1,088 1,088 1,088 1,088 
Adjusted R-squared 0.15 0.19 0.55 0.58 


Standard errors in parentheses. * p < .05,** p < .01 
Baseline: assimilative. 
Source: Adapted from Nguyen et al. (2017). 
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workload, they were set as the baseline. Therefore, the following results should be 
interpreted relative to assimilative activities. 

Table 14.6 shows that assessment activities were positively and significantly 
related to the average time spent in the VLE per week in all four models (Nguyen 
et al., 2017). In Models 1 and 2, the effect of assessment activities was almost the 
same (B = 4.98, SE = 0.57, p < .01 and B = 5.09, SE = 0.59, p < .01, respectively). 
The effect of assessment activities became smaller in Model 3 and Model 4 when 
differences between modules were taken into account. On average, an additional 
hour allocated for assessment activities was associated with 2.47 (SE = 0.47, p < .01) 
and 2.80 (SE — 0.47, p « .01) minutes increase in the average time spent on theVLE 
per week in Model 3 and Model 4, respectively. 

Communication activities were also positively associated with the time spent on 
VLE per week in all four models. For every hour increase in communication 
activities, the time spent on VLE per week increased by 17.44 minutes (SE = 2.11, 
p <.01).The effect of communication activities was the strongest amongst all other 
learning activity types. Interactive activities were positively correlated with time 
spent on VLE in Model 1 and Model 2. However, the effect of interactive activities 
became non-significant when the differences between modules were taken into 
account (Model 3 & Model 4). 

Overall, LD activities explained up to 58% of the variability in student engage- 
ment in the VLE per week when controlling for the heterogeneity between mod- 
ules. To further explore the relationship between LD and student engagement. We 
visualised two exemplary modules in Arts and in Languages (Figure 14.7). These 
two modules had a relatively similar design but the level of VLE engagement 
seemed to be very different. In the Arts module, we can see a peak in VLE activity 
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Figure 14.7 Learning design and VLE activity of two modules in Arts and 
Languages. 
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in week 8-9 due to the increase in workload. The level of engagement then 
decreased during the Christmas breaks and sharply increase just before the Easter 
break in week 26. In contrast, the level of engagement in the Language module was 
relatively constant throughout the module, with the exception during Christmas 
breaks. The level of engagement in both module slightly increased in assessment 
weeks, which confirmed the findings from the fixed-effect models. 


14.5 Study patterns and academic performance 


When teachers design for learning, they often estimate the workload of each activity 
and the corresponding time period for each activity (e.g., take 3 hours to read chap- 
ter 2 in week 2). LD is often embedded in the course syllabus and acts as a guideline 
for students to self-regulate their learning process (Biggs & Tang, 2007). However, 
students as agents consciously and perhaps opportunistically make decisions on 
what, how, and when to engage in a particular range of learning activities (Winne, 
2017). While teachers might think that a student will read say chapter 2 in week 2, 
perhaps some students are already pre-reading materials from week 4, while other 
students may not have watched the introduction video of week 1. Therefore, by hav- 
ing a better understanding of how much time students spent on respective learning 
materials and when in time they studied these learning materials, this may enhance 
our intertemporal understanding of how students make complex study decisions. 

Nguyen et al. (2018b) investigated how students study patterns compare to the 
initial study regime produced for the LD, together with how different groups of 
performance and LD were related to these study patterns. The analyses were con- 
ducted using trace data from the VLE longitudinally over 28 weeks, with 387 
participating students, and replicated over two semesters in 2015 and 2016. Two 
types of study patterns were computed which capture how much time a student 
spent on studying a particular study material: 


e in advance — material x assigned to week t was studied during or before week t 
e catching up or revise — material x assigned to week t was studied after week t 


Overall, given the same study materials, the passed and the excellent group of stu- 
dents spent more time on studying in advance and catch up than the students who 
failed in both semesters (Figures 14.8 & 14.9). In Fall 2015, passed and excellent 
students spent on average each week 1.81 hours (SD = 3.43), and 2.3 hours (SD = 
3.52) on studying in advance, compared to students who failed with an average of 
0.22 hours (SD = 1.05). Similar trends in the time studying in advance across the 
three groups were also presented in Fall 2016. In Fall 2015, passed and excellent 
students followed a similar pattern studying in advance. However, in Fall 2016 
passed and students who failed portrayed a similar pattern for all study materials 
from week 1 to week 12. From week 13 onwards, passed students spent more time 
studying in advance than students who failed. A lot of time was spent on studying 
in advance in weeks 8, 18, and 27 (for Fall 2015) because of the respective assess- 
ments (TMAs) in these weeks (Figure 14.8). 
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Figure 14.8 Number of hours spent on studying in advance. 
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Figure 14.9 Number of hours spent on studying catching up and revising. 


Two study materials in weeks 9—10 (block 2.1) and weeks 12—13 (block 2.3) 
represented red-flags of overwhelming workloads since they were associated with 
an increase in both studying in advance and catch up time (Figures 14.8 and 14.9). 
In Fall 2015, passed and excellent students spent much more time to catch up on 
both of the materials, while the gap was smaller in 2016. 

While students who passed consistently spent more time studying both in 
advance and catch up than students who failed, the relative frequencies revealed a 
different picture. In both semesters, all three groups of students spent a similar 
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Figure 14.10 Percentage of time spent on studying catching up. 


percentage of their time studying in advance in weeks which had a TMA (week 8, 
18, 27). However, in Fall 2015 students who failed spent a higher proportion of 
their time on catching up activities (61% on average) than passed (56%) and excel- 
lent students (55%) in almost all weeks (Figure 14.10). 

In Fall 2016, the three groups shared a similar percentage of study time on 
catching up from week 1 to week 12. After week 12, students who failed spent on 
average much higher proportion of their time on catching up activities compared 
to passed and excellent students. Towards the end of the course, the gap between 
failed and passed/excellent students increased considerably (Figure 14.10). 

In other words, the initial visualisations indicated that student engagement on 
VLE was lower than the suggested time spent in LD. High-performing students, 
who achieved a pass or excellent grade shared similar patterns of engagement. 
However, low-performing students spent the least amount of time on VLE and the 
highest proportion of their studying time on catching up and revising activities. 

By having a better understanding of how, when students study on which materi- 
als, and how these behavioural patterns connected to LD, teachers may be in a 
much better position to reflect and adjust their teaching practices. By explicitly 
pointing out which study materials were under or over-used, teachers can act on 
these materials. This information can be fed back into an LA dashboard, which 
would support teachers and learning designers to track how the students pro- 
gressed through each individual study material (on-track or lagging behind). 
Teachers can use this information to adjust the study workload and re-arrange the 
structure of learning activities accordingly. 


14.6 Discussion and moving forwards 


The increasing development in online and distance education has provided 
researchers with an unprecedented amount of data generated by both students and 
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educators (Shum, 2012). By utilising digital traces of students' activities combined 
with information about the course design, Chapter 14 unpacked some of the tem- 
poral characteristics of how teachers design for learning and how LD influences 
student engagement in distance education. Findings from Chapter 14 showed that 
assimilative activities, such as reading, watching, and listening, were predominantly 
used at the beginning and throughout the module. Assimilative activities were 
often accompanied by productive activities which required students to reflect on 
the information they assimilated. Furthermore, the study workload varied from 
modules to modules and fluctuated considerably from weeks to weeks. 

These findings highlighted a mismatch between educational literature, institu- 
tional policies, and the actual LDs. Previous research suggested that a balanced and 
consistent study workload is essential to student success (Bowyer, 2012; Whitelock 
et al., 2015). However, in practice the study workload is difficult to estimate due to 
the quantity and variety of learning activities used by teachers. The effect of incon- 
sistent workload could be even more detrimental to OU students because the 
majority of them are engaged in either a full-time or part-time employment in 
parallel to studying. By visualising LD on a week by week basis, teachers can reflect 
on the (im)balance of workload in their module and improve their LDs. 

Chapter 14 showed that LD plays an important role in LA research by not only 
improving its model accuracy by taking into account the contextual heterogeneity 
across modules but also offering concrete and actionable feedback to teachers. The 
level of engagement on VLE by students was lower than the expected workload by 
teachers in most weeks throughout the course timeline. What is more important was 
LA models informed by LD can help teachers to identify problematic learning 
activities in which students spent an excessive amount of time revisiting or when did 
students start to fall behind the course timeline. We need to go beyond this kind of 
simple prediction, although it is accurate, to ask real important questions, such as 
which concept or learning activity student X was struggling with, and since when 
student X started falling behind on these activities. Other specific questions about 
the LD could be asked, such as whether students engaged in material X, how long 
students spent on learning activity Y, and how often students revisited concept Z. 
The type of fine-grained analysis illustrated in this thesis allows researchers to ask 
and answer some of these meaningful questions. 

However, there are a lot unknown about the changes in LD practices and stu- 
dent engagement over a longer period of time, such as semesters or years. Therefore, 
future research should consider extending the longitudinal design, such as examin- 
ing the changes in LD of the same module over different semesters, the changes in 
engagement pattern of the same student as they progressed through different LDs, 
and the changes of LD in the same discipline/qualification. 

Furthermore there is a lack of studies on how LD-informed LA could be ben- 
eficial to students learning progress. For example, recommendation systems could 
be built based on patterns of engagement of the previous cohort of students to 
support the subsequent cohorts. Students could use insights from the previous 
cohort to plan and self-regulate their own learning process such as how much time 
should they expect to spend on certain learning activities, which concept that 
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students from previous years struggled with, and how high-performing students 
engaged throughout the course. 


14.6.1 Implications for practice 


Chapter 14 has shown that while student engagement was largely driven by LD, 
there are many potential misalignments between what teachers think students do 
and what they actually do. By having a "reality check" based on actual student 
behaviour, teachers can identify potential problems in their LD and make appro- 
priate adjustments. It is important to have frequent check-ins with students not 
only on the assignment deadline but throughout their learning process. It might be 
too late to intervene by the time a student appears as “at-risk” on the LA system. 
LA should go beyond simple click count with more fine-grained metrics such as 
the duration student spent on each learning activity and whether students are on- 
track or falling behind. This type of fine-grained analysis tightly linked to LD 
could provide important new insights to teachers and may help them to effectively 
intervene where necessary. 
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Chapter |5 


UDL and its implications in MOOC 
accessibility evaluation 


Francisco Iniesto and Garron Hillaire 


15.1 Introduction 


Assessing the accessibility of online educational environments is not easy. As we 
scale up educational technologies like with Massive Open Online Courses 
(MOOCS), we may amplify the impact of (in)accessible designs (Martin, Salvatierra, 
& González, 2016). With MOOCs, we reach more learners and need to consider 
the variability that comes with scale (Papathoma et al., 2020), as previously high- 
lighted in this book (Chua, 2022; Conde Gafaro, 2022; Iniesto, McAndrew, 
Minocha & Coughlan, 2022; Rizvi, Rienties, Kizilcec, & Rogaten, 2022). While 
the challenges and opportunities online learning at scale faces in terms of accessi- 
bility are great, the support from educators and technical experts of the platforms 
that host such courses has historically been minimal (Sanchez-Gordon & Lujan- 
Mora, 2017). 

As indicated in Chapter 9, Iniesto et al. (2022) implemented an accessibility 
audit to understand how to improve the accessibility in MOOCs for learners with 
accessibility needs from an expert evaluation perspective. The audit was conducted 
to help understand the current state of accessibility in MOOCs (Iniesto, 2020). The 
methodology in the audit combined accessibility evaluation methods across four 
main evaluation areas to compose four different checklists in a common heuristic 
evaluation approach: technical accessibility, user experience (UX), quality and 
learning design evaluations. 

Universal Design for Learning (UDL) was selected for the learning design com- 
ponent because UDL offers benefits for both learners and educators in its imple- 
mentation in higher education. The greatest benefits of UDL implementations are 
often experienced by those learners with accessibility needs. While implementing 
UDL there is a reduction in the need for and time required to arrange individual 
accommodations, it is a proactive design that supports a diverse accessibility needs, 
and it offers a greater opportunity to develop more self-aware and knowledgeable 
learners (CAST, 2018). UDL has a thorough development and widespread use in 
primary, secondary and further educational contexts worldwide and it is already 
included in the curricula at university level (e.g., Gronseth et al., 2019). UDL is 
used in educational international initiatives such as INCLUDE (The International 
Collaboratory for Leadership in Universally Designed Education) (Bracken & 
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Novak, 2019) and UDLL (Universal Design for Learning — License to Learn) 
(Griful-Freixenet, Struyven, Verstichele, & Andries, 2017). The most recent Inclusion 
and Education report (UNESCO, 2020) assesses the progress towards Sustainable 
Development Goal 4 which ensures inclusive and equitable quality education and 
promote lifelong learning opportunities for all, drawing attention to all those 
excluded from education because of background or ability, naming UDL 16 times 
in the report. 

Lifelong learning and opportunities to all provided thanks to the implementa- 
tion of UDL in online learning embraces the open world learning theme as an 
enabler that empowers creating expert educators and learners who are critical of 
the learning resources and platforms. To support the long-term aim of developing 
learners as expert evaluators of their learning environments, we focus first on the 
development of a rubric to evaluate MOOCs based on the UDL guidelines. In 
Chapter 15, we detail the reflective process followed to update the UDL frame- 
work, designed for developing accessible educational resources, to an evaluation 
checklist for MOOCS and the implications this process has for extending the aim 
of UDL from creating expert learners to creating expert evaluators. While we do 
not believe UDL can be simplified solely to a checklist we view this as a tool that 
can help to close the loop between the intention of course designers with evalua- 
tions from learners. With that goal in mind, we report on adapting the UDL guide- 
lines to an evaluation checklist, the authors of Chapter 15 computed inter-rater 
reliability using Cohen's Kappa to validate the checklist as an iterative process for 
improvement (Gwet, 2014). 


15.2 UDL to evaluate MOOCs 


UDL offers a framework that considers how to design learning environments to 
develop expert learners, defined in this framework as resourceful and knowledge- 
able, strategic and goal-directed, purposeful and motivated (CAST, 2017). UDL 
favours the elimination of physical, sensory, affective, and cognitive barriers to 
access, learning, and participation of learners (Meyer, Rose, & Gordon, 2014). It 
aims to use various teaching methods to remove barriers to learning (understood 
as anything that prevents learners from fully engaging in learning) and give all 
learners the same opportunity to achieve their learning goals. UDL is based on 
three principles, as further explained in Table 15.1: 


1 Provide multiple means of engagement. Learners differ in how they may feel 
involved and motivated to learn. Therefore, it is necessary to offer options that 
reflect the interests of learners, strategies to face new tasks, choices for self- 
evaluation and reflection on their expectations. 

2 Provide multiple means of representation. Learners vary in the way they perceive 
and understand the educational content. Therefore, it is necessary to offer dif- 
ferent options to approach materials through various channels of perception, 
be it auditory, visual, or motor, so it is required to provide the information in 
a format that allows as much as possible to be adjusted by the learner. 
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Table 15.1 UDL Principles and guidelines 


Provide multiple 


Provide multiple 


Provide multiple 


means of means of means of action and 
engagement representation expression 
Access (7) Provide (1) Provide options (4) Provide options 
options for for perception for physical 
recruiting action 
interest 
Build (8) Provide (2) Provide options (5) Provide options 
options for for language & for expression & 
sustaining effort symbols communication 
and persistence 
Internalise (9) Provide (3) Provide options (6) Provide options 
options for for for executive 
self-regulation comprehension functions 
Goal Purposeful and Resourceful and Strategic and 


motivated 


knowledgeable 


goal-directed 


Source: Adapted from CAST (2018). 


3 Provide multiple means for action and expression. Learners differ in how they can 
work during learning and express what they know. It is necessary to offer 
varied options for action through materials with which all learners can inter- 
act, facilitate fluent opinions, and seek the stimulation of the effort and the 
motivation towards a goal. 


Since the UDL Guidelines are meant to be informed by both new research and 
feedback from the field, they have been updated several times in the past. As indi- 
cated in Table 15.1, the guidelines start with the principle Provide multiple means of 
Engagement but originally it was Provide Multiple Means of Representation. While the 
principles were reordered the guidelines were not renumbered which is why 
guidelines 7-9 appear first in the framework. UDL has recently informed that 
guidelines are going to be updated again following a community-driven process 
(CAST, 2020). 

The three design principles contain nine guidelines and 31 checkpoints. 
Principles outline the overall goal while the checkpoints provide specific design 
advice that considers accessibility and learning. In the most recent version of the 
UDL Guidelines (CAST, 2018), the guidelines and checkpoints have been further 
organised into access, build, and internalise categories. The "access" category 
includes guidelines that suggest ways to increase access to the learning goal by 
recruiting interest and by offering options for perception and physical action. The 
"build" category comprises guidelines that suggest ways to develop effort and per- 
sistence, language and symbols, and expression and communication. Finally, the 
"internalise" category incorporates guidelines that suggest ways to empower 
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learners through self-regulation, comprehension, and executive function. Each one 
of them shows the maturing process to become an expert learner. 

Therefore, the UDL approach is to present the information in ways that fit 
learners’ needs, rather than requiring learners to adapt to the information (Rose & 
Meyer, 2006). This is useful for those users with learning and attention difficulties 
because it allows them to interact with the materials in several different ways 
(Cook & Rao, 2018). At the same time, UDL supports every learner to choose the 
best path for their learning. This approach is relevant to understand learners who 
may like the curriculum to adjust to their needs. As reported above, one of the 
objectives of UDL is not simply the mastery of content knowledge or new tech- 
nologies, it is the mastery of the learning process where education should help turn 
novice learners into expert learners (CAST, 2017), which aligns itself with MOOCs 
pedagogical perspective where learners are expected to be self-directed in their 
learning (Conde Gafaro, 2022). 

The framework proposed by UDL is intended to produce educational content 
following the principles of UDL rather than being evaluated once those resources 
are provided (Hall, Cohen, Vue, & Ganley, 2015). However, UDL has been proved 
as an appropriate framework to categorise and address barriers to learning for 
online environments (Rao, Ok, Smith, Evmenova, & Edyburn, 2020). Following 
the UDL objective to promote expert learners the process of using the UDL 
framework to evaluate MOOCs is an exercise to show the ability to evaluate the 
online environment for barriers to learning. Figure 15.1 shows that the process of 
the application of the framework as an evaluative framework for MOOCs is a sup- 
port for those learning about technology accessibility and learning. Educators and 
learners can benefit from evaluating them in a process that facilities exploring their 
educational needs as expert evaluators (1.e., expert learners). 


15.3 The design and validation of the framework 


The design and validation of the framework fell into three phases. The first phase 
was the creation of the checklist by the first author. The second phase was review- 
ing the checklist with the UDL expert, in other words, the second author. Finally, 
the third phase involved an inter-rater reliability agreement protocol between both 
raters. 


15.3.1 Creating the framework 


The meta-framework proposed included four components (Iniesto, McAndrew, 
Minocha, & Coughlan, 2019). The MOOC accessibility audit components are as 
follows: 


1 Technical accessibility evaluation. Checking of conformance to guidelines 
through WCAG (2018) and the text-based files. 

2 User experience (UX) evaluation. The evaluation of usability and UX charac- 
teristics of the user interface design and pedagogical design. 


UDL principles 


Provide multiple means of 


Engagement 


Affective Networks 
The "WHY" of Learning 


Provide options for 
Recruiting Interest 7 
* Optimize individual choice and autonomy (7.1) 


* Optimize relevance, value, and authenticity (72) 
e Minimize threats and distractions (7.3) 


Adapted evaluation framework 


Guideline 
and What to test for Testing method NA) PK FA 
Checkpoint 

Learners should be able to choose their 
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d UOS of challenge, practising and assessing skills, n 2 a P ^ P. E h 
individual tools used for information gathering or feet Cetin mine ojo a 
choice and production in assignments, deciding the time MOOG, to choose the timing for 
autonomy for completion of tasks completing the educational resources or 
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Figure 15.1 UDL evaluation process. 
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3 Quality evaluation. Evaluation of MOOCs properties, the quality of the 
design, platform, and support for learners. 

4 Learning design evaluation. Evaluation of the learning design characteristics 
within MOOCs using UDL. 


All of them share a standard set of characteristics: checklists were applied to evalu- 
ate the MOOC once it was run and had the same structure (principle, guideline, 
and criteria). Some further characteristics included: 


e The checklist shares the same structure for every single criterion: 
e What to test for: information to help the reviewer to know what the 
criterion is evaluating. 
* Testing method: information to help the reviewer to proceed to test the 
criterion. 
* Comments: space for the reviewer to add free comments. 
e The checklist shares the same rating method: 
e NA (Not achieved): The feature to test is missing. 
e PA (Partially achieved): The feature to test is available but not integrated. 
e LA (Largely achieved):The feature to test is available and partially integrated. 
e FA (Fully achieved): The feature to test is available and fully integrated. 
e  [fthe criterion is not applicable, Not Applicable is added to the comments 


The use of “What to test" for and “Testing method" was based on the accessibility 
heuristic evaluation template by the Inclusive Design Research Centre (Pererya, 
2016). The four evaluation criteria were taken from the OpenUpEd quality label 
benchmark (Rosewell & Jansen, 2014). As Brajnik, Yesilada, and Harper (2010) 
claimed, heuristic evaluations can be complex and even produce wrong results 
(false positives). For that reason, a good definition of what should be evaluated and 
how the evaluation should be carried out has been developed to help the evaluator. 
In the “Testing method” the explanation provided aims to reinforce what a fully 
achieved item would be. Developing a rating system that has four different values 
seeks to avoid a system that considers that a criterion is only fulfilled or not ful- 
filled, by adding the nuance that a criterion can be partially or largely imple- 
mented, in addition the use of comments allows the opinion of the evaluator to be 
explained to enrich the feedback available. 

In the case of the learning design component reported in Chapter 15 the first 
author produced a draft for each of the 31 UDL checklists, named criteria in the 
audit. The design of the criteria for this component required a completely new 
development in the sections “What to test for" and “Testing method”. For that 
purpose, the first author used UDL examples proposed by CAST, which helped to 
prepare specific cases in MOOCs (CAST, 2016). 


15.3.2 Designing the framework 


The design and validation processes between the two authors included a total of 
five meetings (Figure 15.2). That had two aims: improving the component 
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Inter-rater reliability 


meetings 
1 2: 3 4 5 
Framework FutureLearn | edX Canvas  Coursera 


2017-2018 


Figure 15.2 Design and validation meetings. 


specifications iteratively and training to be critical using UDL (i.e., become an 
expert evaluator). Therefore, it was an iterative process between the two authors, 
whereby the second author acted as the rater for the validation of the design. The 
two initial meetings served different purposes: the first meeting had the objective 
of setting the evaluation framework reviewing the checklist proposal. The second 
meeting was used to carry out a process of testing the checklist on a MOOC of 
FutureLearn (Table 15.2). The sample for the validation process for the three 
remaining meetings included MOOCs from Coursera, edX and Canvas. The rea- 
son for the four differentiated platforms was to enrich the sample since different 
providers have different MOOC offering and pedagogical designs, as well all 
MOOCs cover differentiated subjects to allow finding varied barriers to learning. 


15.3.3 Validating the application of the framework 


In each of the validation meetings, the content of “what to test for" and “testing 
method" sections of the criteria were reviewed and improved to evolve from gen- 
eral testing to MOOCs structure. The updated version was used in the next evalu- 
ation. Figure 15.3 shows the evolution of criteria 9.1 and 2.2 as examples from 
meetings 4 to 5 (final version). 

To allow validation of the framework, inter-rater reliability was computed. 
Inter-rater reliability, also known as inter-rater agreement, applies a score of how 
much consensus there is in the ratings given by various raters to test validity 
(Hallgren, 2012). Inter-rater reliability is useful in processes to determine if a scale 
is appropriate for measuring a variable. In the case of rater disagreement, it need 
not follow that the scale proposed is defective but that, for example, the raters need 
more training and better requirements (i.e., the process of becoming an expert 
evaluator through the improvement of the evaluation framework). 

The inter-rater reliability protocol followed a systematised process that included 
two steps per inter-rater reliability meeting. The first step consisted of a brief talk 
before the individual evaluation to agree on the checklist version to be used and 
the course sample (i.e., webpages and educational resources from the platform 
and course) were understood by the raters. In the second step, the results of the 


Table 15.2 Sample for the validation process 


Order Platform provider MOOC Course provider Subject Validated 

| FutureLearn Smart Cities The Open University Engineering No 

2 edX Introduction to Computational MIT Mathematical Yes 
Thinking and Data Science Sciences 

3 Canvas Biometric Technologies: Canberra Institute of Biological sciences Yes 
Identification for the Future Technology 

4 Coursera Learning How to Learn: University of California Education Yes 


Source: Adapted from Iniesto (2020). 
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1. Intest and quizzes, you can find feedback depending T. zs 
lon your answers that helps learner Test and quizzes provide automated feedback, as well, 


9.1 Promote expectations 2- There are examples to show variable goals that can be there are examples to show variable goals that can be 


achieved for learners with different strengths and achieved for learners with different strengths and 
weaknesses weaknesses, learners can reflect in private space to take 


3. The learner can reflect by himself in a private space to Dates CES IEGUU acus co De iin es (One ie 


take notes or discuss with a single peer in privacy one) 


and beliefs that optimise 
motivation 


1. The structure of the MOOC every week is similar and The structure of the MOOC content is similar and keeps 
2.2 Clarify syntax and keeps the same style and structure, keeping the same style and structure, keeping consistency with the 
structure consistency with the terminology m terminology. The content delivered is consistent across the 

2. Atthe beginning of the week there are connections to course modules (e.g. using the same syntax for concepts 


the previous weeks content and activities like multiplication across the module) 


Figure 15.3 UDL checklists / Learning design criteria testing method evolution. 


evaluation and possible improvements were discussed. In case of discrepancy, an 
agreement was then reached between the evaluators on how that particular crite- 
rion should be qualified (Cook et al., 2009). 

The terminology that describes the agreement process includes: 


e Success and failure. Success criteria are considered fully achieved (FA) and 
largely achieved (LA), failure criteria are considered partially achieved (PA) 
and not achieved (NA). 

e Agreement. Evaluators can agree that is either perfect agreement or moderate 
agreement. 

* Perfect agreement. A perfect agreement is considered when the evalua- 
tors gave the same rating to a criterion (e.g., FA vs FA, NA vs NA). 

e Moderate agreement. Moderated agreement is when the rating given by 
the evaluators differs within the same group (success or failure). 

e Disagreement. The rating between the evaluators differed between success or 
failure (e.g., PA vs FA or NA vs FA). 

* Final agreement. The final rating as discussed and agreed. 


We included statistical mathematical agreement using Cohen's Kappa (k) when 
examining the agreement between two raters (0.01—0.20 slight agreement, 0.21— 
0.40 fair agreement, 0.41—0.60 moderate agreement, 0.61—0.80 substantial agree- 
ment and 0.81—1.00 perfect agreement) (Gwet, 2014). 

The edX agreement was of 24 out of 31, with 10 full agreements (k — 0.55). 
With Canvas there was a consensus of 24 out of 31, 11 of full agreement (k = 0.55). 
Finally, with Coursera, there was a concurrence of 26 out of 31, 12 of the criteria 
with full agreement (k = 0.59) (Table 15.3). Overall, the results indicated that the 
level of agreement increased during the third round of evaluation. In all cases, for 
the full evaluation process Cohen’s Kappa showed a moderate agreement. When 
looking at the principles there were agreement fluctuations even when the specifi- 
cations of the criteria included in the checklists were improved at each iteration. As 
well, there was variation at the criteria level in each iteration, which might be due 
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Table 15.3 Cohen's Kappa between evaluators 


Engagement Representation Action and Full evaluation process 


expression 
edX 0.28 0.83 0.57 0.55 
Canvas 0.19 0.47 0.57 0.55 
Coursera 0.8 0.42 0.39 0.59 


to the nature of the sample, with different MOOCs and providers, and the richness 
of specifications in the UDL criteria. 

To understand where the raters were improving their agreement and areas 
where it was more difficult, we further disaggregated the analysis at the principle 
level. Tables 15.4, 15.5 and 15.6 show the agreements and disagreements including 
both moderate agreements and final agreement. These tables are helpful to show 
visually the discrepancies during the process. Only one criterion of the 31 had not 
incurred any discrepancies (“Use multiple tools for construction and composi- 
tion", 5.2), which exemplified the different interpretations that can be made while 
learning how to internalise UDL. Highlighted disagreements in the following 
explanations included those who have appeared in two or more evaluations. 

For the principle *means of engagement" (Table 15.4), those criteria where 
there had been a substantial discrepancy were: 


e “Optimise individual choice and autonomy" (7.1) and “Optimise relevance, 
value, and authenticity" (7.2). It was difficult to evaluate if learners can choose 
their challenges while participating in discussions and assignments and if those 
allow learners to connect with their learning goals. 

e “Foster collaboration and community” (8.3). Difficulties when evaluating 
arose to determine if learners can find support to foster collaboration to 
encourage discussions. 

e “Facilitate personal coping skills and strategies" (9.2). In a MOOC environ- 
ment, it was not straightforward to identify places for learners to identify cop- 
ing skills in discussions, either originated by facilitators or learners themselves. 


Therefore, in MOOCs it is difficult to evaluate the engagement between learners 
and educators (and learners themselves) in discussions and assignments, an aspect 
reported both in the literature (Bote-Lorenzo & Gómez-Sánchez, 2017) and in this 
book (Chua, 2022; Conde Gafaro, 2022; Rizvi et al., 2022), losing part ofthe MOOC 
claimed support for social interaction (Sunar, White, Abdullah & Davis, 2016). 

In the case of “means of representation” (Table 15.5), at criteria level key dis- 
agreements were: 


e “Highlight patterns, critical features, big ideas, and relationships" (3.2) and 
“Maximise transfer and generalisation” (3.4). It was not easy to identify exam- 
ples of essential features, places where having previous knowledge could help. 
As well, the existence of tools for the learners to personalise and generalise 
their learning. 


Table 15.4 Agreement process for "provide multiple means of engagement" 


Criteria X checklists edX Canvas Coursera 
RI R2 FA RI R2 FA RI R2 FA 

7.| Optimise individual choice and autonomy PA NA NA LA PA PA PA LA LA 
7.2 Optimise relevance, value, and authenticity LA PA PA LA PA PA FA LA LA 
7.3 Minimise threats and distractions LA LA LA LA LA LA LA LA LA 
8.1 Heighten salience of goals and objectives PA NA NA PA PA 

8.2 Vary demands and resources to optimise challenge NA NA NA NA NA NA LA FA LA 
8.3 Foster collaboration and community LA NA PA LA PA LA LA FA LA 
8.4 Increase mastery-oriented feedback NA NA NA FA LA LA FA NA NA 
9.| Promote expectations and beliefs that optimise motivation PA NA PA LA PA PA PA 

9.2 Facilitate personal coping skills and strategies LA NA PA LA PA PA PA 

9.3 Develop self-assessment and reflection PA NA PA PA NA 


Note: RI = Rater |; R2 = Rater2; FA = Final Agreement; NA = Not Achieved; PA = Partially Achieved; LA = Largely Achieved; FA = Fully Achieved. 


Source: Adapted from Iniesto (2020). 
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Table 15.5 Agreement process for "provide multiple means of representation" 


Criteria X checklists edX Canvas Coursera 

RI R2 FA RI R2 FA RI R2 FA 
1.1 Offer ways of customising the display of information LA FA LA PA PA LA PA 
1.2 Offer alternatives for auditory information FA LA LA PA LA FA LA 
1.3 Offer alternatives for visual information PA PA PA NA PA PA LA FA LA 
2.| Clarify vocabulary and symbols PA NA NA NA LA FA LA 
2.2 Clarify syntax and structure FA LA LA PA LA FA LA 
2.3 Support decoding of text, mathematical notation, and symbols PA NA NA Not applicable Not applicable 
2.4 Promote understanding across languages PA NA PA PA NA NA FA LA LA 
2.5 Illustrate through multiple media NA PA PA LA PA LA PA 
3.1 Activate or supply background knowledge PA PA PA LA PA PA LA FA LA 
3.2 Highlight patterns, critical features, big ideas, and relationships FA PA PA LA PA PA FA LA LA 
3.3 Guide information processing, visualisation, and manipulation LA FA FA LA LA 
3.4 Maximise transfer and generalisation FA PA PA LA PA PA PA 


Source: Adapted from Iniesto (2020). 
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Table 15.6 Agreement process for "provide multiple means of action and expression" 


Criteria X checklists edX Canvas Coursera 
RI R2 FA RI R2 FA RI R2 FA 
4.| Vary the methods for response and navigation LA FA LA LA NA 
4.2 Optimise access to tools and assistive technologies LA FA LA LA NA LA PA 
5.| Use multiple media for communication LA NA PA LA PA NA PA 
5.2 Use multiple tools for construction and composition LA PA PA 
5.3 Build fluencies with graduated levels of support for practice and PA NA NA LA PA PA PA 
performance 
6.| Guide appropriate goal-setting LA NA PA PA PA NA PA 
6.2 Support planning and strategy development NA NA NA LA LA 
6.3 Facilitate managing information and resources PA NA PA NA PA PA NA PA PA 
PA NA PA LA LA 


6.4 Enhance capacity for monitoring progress 


Source: Adapted from Iniesto (2020). 
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In MOOCs information about prior knowledge should be clear before 
enrolment and there should exist pointers to external resources during the course. 
As well, there is a lack of possibilities for personalisation, something that is 
problematic at the platform level since it affects courses even if they have been 
designed allowing space for personalising the learner experience (Garcia-Penalvo, 
Fidalgo-Blanco, & Sein-Echaluce, 2018). Finally for “means of action and 
expression" principle (Table 15.6), criteria showed disagreements at some stage, but 
they were not repeated across the several evaluations. 

The fact that the “means of engagement" principle accumulated more disagree- 
ments than “means of representation” but still more than “means of action and 
expression" pointed towards engagement complexities in MOOCs. Expert learn- 
ers in MOOCs are often strategic and goal-directed, therefore, they are good at 
self-directed learning (Watted & Barak, 2018), but do worse in being purposeful 
and motivated, which is aligned with the drop-out rates and difficulties to keep the 
engagement in MOOCs (Petronzi & Hadi, 2016). 


15.4 Discussion and moving forwards 


In Chapter 15 we have reported on the adaptation of the UDL framework for 
evaluation purposes following an inter-rater reliability validation process. UDL 
evaluation processes by educators and learners facilitate the empowerment of cre- 
ating expert learners who are critically evaluating the educational platforms and 
resources. Discussions during the validation process between the two raters 
included multiple conversations on what to evaluate in MOOCs using the UDL 
framework. In that sense, one of the principal difficulties was to distinguish whether 
the requirements should be addressed at platform or course level. This aspect is 
identified where the reliability agreements were lower which is underpinning dif- 
ferences between the platforms s and their individual features for hosting educa- 
tional resources and underlying pedagogical models. That aspect is linked with the 
lack of specification of MOOC completion and inclusion of discussions in educa- 
tional resources, the lack of options to optimise individual choice and ways of 
customising the display of information. Aspects already reported in the broader 
literature of learning at scale in MOOCs (Joksimovic et al., 2018). 

A key aspect of the UDL checklist after its validation was its complexity. Criteria 
in the UDL checklist were often rich, covering multiple aspects to be evaluated, 
which made it complicated for an evaluator to decide a final rating. That was linked 
with the fact of the complexity of acquiring expertise in UDL, and therefore an 
expert evaluator through the MOOC evaluation process. Future evaluations should 
consider including more MOOCs per platform provider, and more platforms, to 
understand the different evaluations between course providers using the same plat- 
form. In a broader context, further research in learning design and accessibility in 
MOOGCs should also consider other aspects such as the role of learning analytics to 
understand the diversity of learners needs (Cooper, Ferguson, & Wolff, 2016), how 
learner-emotions affect learning (Hillaire, Iniesto, & Rienties, 2019) and how per- 
sonalisation informs the learning process and engagement (FitzGerald et al., 2018). 
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The process in Chapter 15 has shown that UDL cannot be stored in a box and 
applied as a simple checklist. The design and validation of evaluation checklists 
indicate a complexity of learning, whereby different concepts derived from the 
UDL framework proposed and how practising in this iterative evaluation and vali- 
dation process helps to internalise UDL principles and guidelines. But as well 
showcasing the appearance of complex concepts to evaluate or redundancies that 
might be creating difficulties in that mastering process. 


15.4.1 Implications for practice 


When producing educational resources all stakeholders should be considered in 
their design and evaluation processes including educators and learners. To all 
those educators interested in universal and inclusive design, reflecting on the 
UDL principles and guidelines may help when producing educational resources 
and critically evaluating them, which may facilitate and foster inclusion and 
widen access. To make online courses more accessible it is necessary to put in 
place processes to identify barriers to learning and strengthen mechanisms that 
facilitate agile responses in addressing those barriers. Regardless of whether to 
design MOOCs or any other online course, it is necessary to consider the differ- 
ent platforms that exist and how their pedagogical designs influence our deci- 
sions. In this sense, they can affect aspects of UDL such as the production of 
educational resources in alternative formats, the promotion of interaction and 
engagement between learners or the inclusion of space to allow reflection and 
personalisation of the learning experience. 
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Chapter 16 


Practitioner's perspective on 
young children's use of mobile 
technology 


Pinsuda Srisontisuk 


16.1 Introduction 


The 21st century and its technological advancement, particularly mobile technol- 
ogy, have brought about a new outlook towards teaching and learning in the early 
years. Whether children are directly interacting with these new devices or merely 
exposed to it, it becomes evident that most "rapid changes in these technologies 
create a new landscape of knowledge, leaning and growing up for young children" 
(Arnott, 2017). Kabali et al. (2015) emphasised the universality of exposure to 
mobile devices. In their research, they found that the pervasiveness of technology 
towards young children was evident across different social levels, irrespective of 
income or race. 

In Chapter 16, we focus specifically on mobile touch screen technology (MTST) 
as their unique features of being mobile and touchscreen has played a role in the fast 
adoption of these devices amongst young children. The growing trend of young 
users is visible, not only through data collected from national surveys (Ofcom, 2019), 
but it is also evident in the sheer growth of apps targeted towards pre-school chil- 
dren internationally, with apps for this age group dominating the education app 
category at 58% (Shuler, Levine, & Ree, 2012). A more recent study investigated 
how beneficial these “educational” apps specifically targeted at young children, the 
results of analysing over 100 apps, indicated that the apps were not as educational as 
expected (Meyer et al., 2021). As a result, the pervasiveness of MTST in young chil- 
dren' lives raises the question of how technology should be introduced and focused 
on early years practitioners’ perspectives towards these new devices. 

This main research question that drives the narrative of Chapter 16 is “What 
attitudes and opinions do practitioners have towards young children's use of mobile 
touch screen technology?". Chapter 16 will provide an overview of previous 
research that has looked at early years practitioners’ beliefs and attitudes towards 
technology and young children, acknowledging the limited research in this field. It 
should be noted that the findings from Chapter 16 are based on empirical data 
from a larger design-based research project that aimed to investigate how children 
cooperate when using tablet computer in pairs. For the purpose of Chapter 16, 
when referring to young children the specific age group that we are discussing are 
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children aged 3-5 years old, as this is the age group in which practitioners were 
working with. 


16.2 Research concerned with early years practitioners' 
beliefs and attitudes towards MTST 


Multiple studies have explored how teacher beliefs and attitudes towards technol- 
ogy influence adoption and usage, especially in relation to primary school-age 
children (Ertmer, 1999; Ifenthaler & Schweinbenz, 2013). However, far less research 
has explored the perspective of pracitioners and teachers of young children, an 
important group given the debate on the place of technology in the lives of these 
young children (Blackwell, Lauricella, Wartella, Robb, & Schomburg, 2013). The 
studies that have explored early years practitioners! perspectives often employed a 
quantitative approach in the form of surveys (Gialamas & Nikolopoulou, 2010; 
Aldhafeeri, Palaiologou, & Folorunsho, 2016). The findings usually indicated a 
positive attitude towards MTST, although concerns were also expressed, such as 
delays in social development, inapproate material, and addiction (Marsh et al., 
2015; O'Connor, 2017). 

A limited number of more qualitative research and mixed-methods approach 
studies have also been conducted, involving focus groups that offered more perspec- 
tives in terms of the social expectations from their work colleagues (Hatzigianni & 
Kalaitzidis, 2018; Palaiologou, 2016). For example, Hatzigianni and Kalaitzidis (2018) 
found that early years teachers views were evolving, and acknowledged a changing 
trend where practitioners were open minded to these new technologies. Using a 
Bronfenbrenner’s ecological framework Hatzigianni and Kalaitzidis (2018) focused 
on how personal digital skills, leadership styles, teacher training, and teaching phi- 
losophies were associated with a positive attitude towards incorporation of technol- 
ogy. Palaiologou’s (2016) international survey conducted across five countries 
demonstrated that practitioners consistently demonstrated a positive attitude and 
aptitude towards digital devices. However, practitioners appeared hesitant about 
integrating technology in their practice. This reluctance was unpacked in subsequent 
focus groups and was attributed to the fact that “the dominant ideology is that play- 
based pedagogy leaves no space for digital devices to be included” (Palaiologou; 
2016, p. 313). T his conflicting perspective towards digital technology appeared to be 
consistent cross-culturally as the view towards digital devices were not seen as offer- 
ing interactions or experiences that could be considered as playful, and thus part of 
a play-based pedagogy that is the core to most early years curriculum. 

Given that this is a critical age for all aspects of development and that technology 
is an undeniable part of everyday life for these children and will be even more 
eminent as they grow up, educators are concerned about how to establish a “posi- 
tive start” in using technology. Research has shown that “problematic media habits 
may predict a trajectory of increasingly excessive use through adolescence” 
(Radesky et al., 2014, p. 1176) and therefore the early years are fundamental. 

As for the studies that focused on interviews with practitioners, these either 
looked at ICT in general (Mertala, 2017) or specifically at computers (Alkhawaldeh 
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et al., 2017). Therefore, further interview studies to provide more insights into 
practitioners’ perspectives in settings that are users and non-users of MTST are 
needed to better understand the educational context of young children in pre- 
school settings. Understanding how practitioners view MTST technology and 
what promotes or hinders practitioners in the early years sector to adopt any type 
of technology is critical. It is often a precursor to how technology is introduced 
in the primary school level, but it also is a reflection of the gap between practice 
and pedagogy. 


16.3 Research design 


Chapter 16 aimed to gather data about practitioners! perspectives on MTST 
through the use of semi-structured interviews. The participants involved nine 
practitioners from two different early childhood educational settings, one setting 
was a nursery based in Reading (no access to MTST) while the other was a school 
situated in east London (access to iPads). Using purposeful samplings (Palinkas 
et al., 2015), each of the practitioners had varying degrees of experience in teaching 
in the early years, ranging from 1—17 years. These samples are not meant to be 
representative but are intended to provide a holistic view from multiple perspec- 
tives. Interviews lasted between 30 and 50 minutes, were audio-recorded, tran- 
scribed, and analysed through an inductive thematic analysis approach (Braun & 
Clarke, 2006) by the author. Five major themes emerged from asking practitioners 
about their views on young children's usage of MTST: (1) Ubiquity of Technology 
(2) Ability in using MTST (3) Use in Moderation (4) Values of technologies in 
Early Childhood and (5) Negative Aspects of MTST. 


16.4 Research findings 
16.4.1 Ubiquity of technology 


The first theme of ubiquity of technology referred to comments that practitioners 
made regarding how prevalent MTST is in young children’s environment as well 
as in our daily lives as adults. This was also evident in Chapter 4 (Vogiatzis et al., 
2022), whereby a teacher mainly used WhatsApp as a language theme method as 
most people have ubiquitous access to MTST. This theme manifested itself in dif- 
ferent ways. Either the teacher was explicit in saying that MTST is here to stay and 
therefore we must be prepared for it, or through anecdotal stories they explained 
the pervasive nature of it and how we all are becoming regular users of MTST. This 
was an important and major theme as it reflects the changing outlook amongst 
practitioners about the usage of technology. 

It demonstrates a shift in the conversation from the binary question of whether 
(or not) practitioners should be using technology towards a more constructive 
dialogue about how to best to use technology and to what extent it will yield the 
most educational benefit for young learners. This finding supported previous find- 
ings regarding the increased uptake of MTST in the early years as well as an 
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acknowledgement that young children today are growing up in a distinctively dif- 
ferent technological environment than their parents (Karagiannidou, 2017). 
Practitioners in both the London and Reading settings agreed that technology 
should be introduced. However, their tone and further explanation of why it should 
be introduced to children varied across practitioners. A majority of practitioners 
expressed that the need to introduce this type oftechnology was mainly due to how 
prevalent technology is in our society nowadays. As one practitioner mentioned, 


I think it’s a good idea, because the world is moving so fast, I mean new things 
are coming up. If they don’t, if they're not, you know, expose to all of this, then 
they'll find it difficult later on. So it’s best that they actually know how to use 
thing, it helps with their learning as well. 

(P8, London) 


'There appeared to be a general recognition that MTST are becoming widely used 
and thus it becomes essential for early year practitioners to prepare children to use 
these types of technology appropriately. Similar perspectives on technology pres- 
ence were reflected in other practitioners as they agreed that technology should be 
introduced: 


I suppose because this day and age, technology is more in use. 
(P7) 


they just grow up with technology so they always expect it. 
(P4, Reading) 


'The idea of technology presence as a rationale for introducing MTST in the early 
years appeared to be an agreed-upon notion. However, the implication of technol- 
ogy presence, including the perceived benefits and harms varied much more across 
individual practitioners, and will be discussed further in the fifth theme. 

In another example, practitioner 6 shed a light on “ubiquity of technology" by 
making a comparison to her own childhood, and highlighting the differences in 
how time is now spent: 


when I was little you go out and bike in the street and you went to play foot- 
ball, but now children want to sit on the iPad, even my niece she is two years 
old, she can pick up a phone and can open it, she can sit there for hours, which 
I guess as a parent is really useful sometimes, but not all the time, I mean like 
she's been taken to the park and what not but it is concerning that so much 
time is spent on there. 

(P6, London) 


'This reflective thinking and comparison to one5 childhood can often be found in 
research regarding uncertainty around new technology and parenting styles 
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(Plowman, McPake, & Stephen, 2008), and this is also true in teachers’ teaching 
philosophy. It is not uncommon to make these comparisons, as we use our own 
experience as points of reference when working with young children. It becomes 
more challenging when practitioners are working with technologies that have not 
been part of their own childhood experiences. Although practitioner 6 was 
expressing her concerns about the potential harm that technology brings with it, 
she also highlighted another key component of technology presence and how it is 
re-shaping the culture of young children. As children share their day-to-day expe- 
rience with their peers or teachers, stories about their weekend may include ele- 
ments of watching a certain YouTube clip, or playing a new game on the iPad, in 
addition to or replacing more traditional stories of going to the park. This famil- 
iarity of technology has led to children sharing and developing different skill sets 
around technology use. 

The first theme also presented itself in other ways, such as when one practi- 
tioner who was very optimistic about technology and commenting was ques- 
tioned on how she thought the devices offered a lot of possibilities for learning. 
Reflecting on whether she was always receptive to new technology, she replied, 
“I think since I’ve been teaching, it's always been in the forefront. So I can’t 
really compare it to before" (P5, London). This was another perspective on tech- 
nology’s presence and it illustrated how technology was advocated in the educa- 
tional sector. It should be noted that she had experience teaching slightly older 
children, which might have shaped exposure and experience in using technol- 
ogy with children, as she also mentioned a project she did with her older stu- 
dents using stop-motion animation. This perspective might reflect some 
practitioners’ experiences based on their training and school cultures that were 
more proactive in supporting and engaging practitioners in the use of MTST in 
their teaching practices. An unexpected point that emerged from the theme of 
technology presence was how for some children, the presence of all this technol- 
ogy changed the children’s accent, given that children were watching more 
American YouTube clips at home. 


16.4.2 Ability in using MTST 


The second theme "Ability in using MTST" involved comments related to how 
easy, comfortable, and confident many of the children were in using MTST, 
because of how prevalent it is now in their home environment. Many practitioners 
noted that they could not be certain how much usage of MTST the children were 
getting outside of the school, as it was not something they tracked. However, there 
was certain behaviour that could be observed which indicated increased usage 
outside of the school environment. This related to how confident and knowledge- 
able children were when using the devices, prior to any instructions. This was a 
sub-theme as it was derived from the practitioners’ comments that the children’s 
confidence in using technology was the direct result of how much exposure or 
usage they were getting outside of school: 
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Some of the children are, you can tell cos some of them are really confident to 
speak to you about the programs and all the stuff that they use on the iPads, 
like the YouTube videos. They'll say like ‘Oh!, I saw this on the YouTube 
video’. They know a lot of games that they play on the iPad, so that’s how you 
know that they often on it at home. The majority do have access to iPad. 
(P7, London) 


Comments about children’s confidence in using technology were echoed by other 
practitioners. Although practitioners did not address the more intuitive design of a 
touch-screen interface and the user-friendly nature of the devices that made it 
more accessible to young children, they did comment on a gap in expertise, and 
how children appeared to be more adept in using MTST than some adults. As 
practitioner 8 explained: 


These are guys (children) are too good, I think some of them are a bit too 
forward than the adults and they know a bit too much sometimes. I think with 
technology like that smart phones and iPhones they have it at home. So they 
are really equipped, especially the ones who have them readily available. 

(P8, London) 


Practitioners viewed the children’s confidence in using the devices from multiple 
angles: the tone varied from impressed at how adept the young children were at 
using it, to concerns that other essential skills such as holding a pencil were not 
being nurtured at home. An example of a more impressed tone could be seen 
when one of the practitioners listed all the things a child was capable of doing: 


some of them know how to like turn the volume down, how to switch off. 
Yeah some of them, yeah it’s crazy, (they) know how to turn the volume, know 
how to turn it up, know how to brighten up the screen and I’m just like “Well, 
I don’t even know this”. And they know how to get on it as well. 

(P2, Reading) 


As mentioned earlier, some practitioners took a more cautious view that although 
the young children were able to easily navigate through these devices, it might 
come at the cost of not developing other essential skills, such as mark-making and 
writing: 


It’s so interesting that some of them can come in and open the tablet, go 
around you know the different apps, when they’re barely, you know just 
turned 4, but they can’t pick up a pen, they can’t hold a pen to make a mark. 
And I think it’s getting the balance between the two. 

(P6, London) 


Practitioners appeared to be keenly aware of a different skillset that young children 
had developed, which included children using their fine-motor skills to use MTST 
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devices. Being able to swipe and navigate through these tools with confidence, 
although such a skill set might appear to be more intuitive than using a mouse or 
writing with a pencil, was still a different competency that developed as a result of 
more exposure to technology. 


16.4.3 Use in moderation 


The third term use in moderation indicated comments where practitioners talked 
about ensuring that children had exposure to technology but also sufficient 
time to engage in other types of activity. This idea of balanced choices often 
manifested itself in practitioners commenting on the necessity of setting up 
“limits”, and ensuring that children were engaging in other learning activities 
as well. There was some mention around the potential difference in the home 
and school setting, where setting up this balance at school was easier. This 
might not be so easy in the home setting where children may be spending 
substantially more time on the devices than appropriate. Although the theme 
was frequently expressed from the viewpoint of needing to set limitations, the 
term use in moderation rather than limits was used in Chapter 16 as it was a 
more reflective of the idea that practitioners still felt there was a place and time 
for MTST. 

Practitioner 1 explained how she structured it in her setting, and set up clear 
time frames of usage: 


I think that it could be introduced as long as there are some limits, not all day 
long. We are here for half an hour, we switch off the computer. Go to that 
room, so we can enjoy that space, if it’s just something new and the game, 
maybe in the day half an hour morning and in the afternoon. For children to 
do some educational activities. 


(P1, Reading) 


Practitioner 1 went on to explain how she also set limits within her own house- 
hold, and how that was made clear to her own children, “You have to give 
limits, how long they can use it for? You have to give limits. This iPad, for me 
I've got two children, they only use on the weekend.”. Other practitioners also 
expressed similar views on setting up limits, “I think that it would be good to 
introduce with limits and not having certain things on there.” (P3, Reading). 
The idea of making a balanced choice often required the intervention of a 
practitioner to monitor their usage and ensure that certain children were not 
always on the devices: 


I think it’s fantastic, a fantastic learning tool for them. And obviously we limit 
the amount of time they're on. There are some children that would obviously 
love to sit there, like all day long on the iPad, playing games and watching 
videos. So you do try and limit them. 

(P5, London) 
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Theme 3 was naturally linked with Theme 1, and the concern that because the 
devices were so prevalent, children were not engaging in other learning activities. 
Practitioner 6 explained this further: 


I think when they're (MTST) used in the right way for an activity that it's 
benefiting them. But I think it needs to limited so that they're getting other 
experiences, and I think especially we find with our children they do spend a 
lot of time on tablets and mobile phones at home. So we try to limit the time 
that they get on them in school. 

(P6, London) 


16.4.3.1 Value of MTST in early childhood education 


The fourth theme addressed practitioners’ comments about the positive value that 
MTST had in early childhood education. Some comments referred exclusively to 
the three Rs (reading, writing, arithmetic), referencing specific apps that catered to 
these knowledge developments. Practitioners also talked about the positive atti- 
tudes that children had towards learning when using the devices as well as the 
ability for technology to extend their learning. This extension of learning was one 
of the unique features of mobile devices, involving instant access to information 
that these tools now enabled through the Internet. 

The feature of being able to find information online, and research certain con- 
cepts in the present, allowed the option to extend a learning opportunity further. 
This also helped to provide visual stimuli to explore and explain certain new con- 
cepts. Practitioner 5 gave an example from her classroom: 


If someone comes up with a topic of discussion or something they’re inter- 
ested in, I can just get google up, “Oh lets”, you know if you’re interested in 
harp, like Jack and the Beanstalk. Everyone was like what’s a harp?, they 
weren't sure. You know, it was easier to find pictures of harp and then to play 
some harp music them and give them that experience. 

(P5, London) 


The interesting thing about theme 4 was how practitioners went into detail and 
were able to give concrete examples about an incident, where they were able to 
search something online to help show or explain something further to the young 
children. This illustrates that practitioners experienced the advantages and benefits 
of being able to access information at any given time. 


We had a little girl she got onto google maps, and like I showed her where are 
nursery was and then I showed her London and where we live, and that was 
an interesting one. Yeah, and then we showed her the world. The world was on 
there, “We live in England ... there is other people that live here”. She was like 
“Oh! We come and play here, we come to nursery here". So I was like “Yeah”. 
I thought that was pretty clever. 

(P2, Reading) 
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The ubiquitous knowledge access was often viewed in a positive light, and practi- 
tioners expressed how accessibility allows for a more enriching experience for the 
children. Practitioners valued this access to the Internet: 


They do have the internet explorer that they can use if they want to find out 
about something. Usually I go on the iPad and I show them, then they're 
aware of it, so then they can try and type out the word that they research. 
I personally like to use it for research. 

(P9, London) 


The findings demonstrated that many practitioners recognised the potential edu- 
cational benefits of technology. One being the technology connectivity to 
Internet, with the instant access to information being a tremendous asset to their 
teaching in extending the children’s understanding of the world. According to the 
view that early childhood teachers helped to facilitate how children develop their 
culture and foster their ability to practically produce knowledge (Ihmeideh & 
Alkhawaldeh, 2017) this instant access to information has allowed children to 
extend their learning. However, such engagement was often teacher-led. 


16.4.4 Negative aspects of MTST 


The fifth and final theme demonstrated the different types of concerns that prac- 
titioners had. There was a concern that when children were too immersed in tech- 
nology they became too passive, and they did not communicate with others, or 
developed those essential physical, social, emotional skills that are at the core of 
early years education. This subdued interaction with technology was viewed as 
almost an addictive behaviour, and that their engagement with technology was no 
longer productive once it took away time from other essential types of play. 

A fear was expressed by one of the practitioners, "That the children will become 
so technology involved that they're not interested in anything else. They're not giv- 
ing those practical experience of going out to run but now children want to sit on 
the iPad.” (P6, London). Another practitioner explained further why excessive pas- 
sive interaction might hinder their cognitive development, "Fear, is that they may 
become a bit passive, it’s already there they don't have to think much it’s just click, 
click, click. That's what I feel some time, more of that and less of thinking.” (P7, 
London). Practitioner 2 explained her fear when a balanced choice is not made, 
and how technology limits their interactions with others: 


Bad things are I think some of them can just sit and sit and not interact with 

anybody else and there is no talking there. And their communication is just 

with colours, screen, a box. I don’t like them sitting on their for too long. 
(P2, Reading) 


The fifth theme was expected given a general concern that the media presents and 
current observational behaviour in which children appeared to be overly immersed 
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with technology. Practitioners were expressing genuine concern over how the 
devices did not encourage active learning, but instead it was just a passive con- 
sumption of information. 


16.5 Discussion and moving forwards 


Chapter 16 provided an overview of how early child practitioners from two UK 
schools perceived the use of mobile touch screen technology (MTST) by young 
children. As practitioners acknowledged the growing presence of these devices, 
they shared differing perspectives on the educational potential of the MTST and 
the perceived harm, adding to the ongoing debate. The interviews with nine early- 
years practitioners confirmed and extended the findings of previous research, and 
it provided new insights about this group of young children. Practitioners’ perspec- 
tives on how much exposure and usage the children had with MTST, confirms the 
growing statistics around young children’s adoption of these new technologies. 
Similarly, the ease with which young children can use tablets with minimal instruc- 
tion is potential evidence of how much they are using technology outside of school 
as well as the intuitive designs of tablets computers that has allowed for such young 
users to purposefully engage with them. 

The fear around technology of being passive devices and not allowing the young 
children to engage in other social and physical activity is still at the forefront of 
many practitioners. These are the exact same concerns that were put forth earlier 
by early years practitioners about desktop computers (Wood et al., 2008), so the 
mobility of the device and the new software has not been able to adequately 
address this major concern. Practitioners felt that limited use and balancing chil- 
dren' interaction with these devices and other activities was the ideal solution. 
Practitioners also reported that one of the most beneficial features was the con- 
nectivity to Internet; instant access to knowledge and images was a feature they felt 
has enhanced the learning experience for their students. 

Although there were some themes that were presented across both educational 
settings, which highlights the commonality in hopes and concern around MTST 
devices, it should be noted that the tone and emphasis that each practitioner 
expressed on a theme would vary even within the settings. For example, in the 
London school there was one practitioner who enthusiastically commented on all 
the potentials of learning that had arisen because of technology presence, while 
another practitioner was quicker to highlight the addictive behaviour that has 
occurred because of technology presence. The disparity of opinion between prac- 
titioners is a common phenomenon when looking at the impact of technology on 
education that reaffirms previous findings (Guha, 2003). Subtle differences in views 
were to be expected. However, extreme inconsistency amongst practitioners within 
settings may hint at a more substantial problem in which there is not enough evi- 
dence and research to persuade practitioners one way or another. There is a need 
for more research with this age group to provide more evidence to practitioners 
and continue to engage them in the ongoing conversation around what type of 
interaction and engagement with technology are appropriate for young children. 
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16.5.1 Implications for practice 


Chapter 16 suggests that practitioners should seek to understand how children 
engage with MTST to benefit from the ways technology can enhance young 
children's learning experiences. Practitioners' abilities to identify their own con- 
cerns about the usage of MTST in young children should be an ongoing dis- 
course, to prompt innovate ways to integrate and address these worries. Therefore, 
educational settings should also consider setting more explicit and concrete 
boundaries to the use of MTST, and work towards formulating their philosophy 
towards technology that best suits their context so that it can be presented to both 
learners and parents. 
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Chapter |7 


Antecedents and consequences of 
uncertainties perceived by 
finance professionals 


Vasudha Chaudhari, Allison Littlejohn and Simon Cross 


17.1 Introduction 


Uncertainty is an inherent component of the modern-day workplace (Brown 
et al., 2011). It is a major area of interest within the field of management, change 
management, and organisational learning (Bohlinger et al., 2015a; Downey & 
Slocum, 1975; Markowska & Wiklund, 2020). During ambiguous times, profes- 
sionals need to learn to navigate their work lives through various uncertainties, as, 
for example, also illustrated in Chapter 18 (Iwaniec-Thompson, 2022).The ability 
to learn in uncertainty can be considered from two perspectives — individual learn- 
ing that shapes the individual performance (micro perspective) and organisational 
learning that defines organisational success (macro perspective). However, much of 
the existing literature tends to focus on the macro perspective of the organisations 
(see, e.g., Michel & Wortham, 2009; Zhao & Wang, 2020).There are a few examples 
that combine the micro and macro perspective. For example, Wang et al. (2019) 
found that individual unlearning mediated by organisational unlearning (1.e., dis- 
card work practices that are no longer relevant or serve their purpose) and relearn- 
ing (i.e., ability to forget practices gained from old knowledge and relearn new 
practices) has a positive effect on strategic flexibility. This means that strategic flex- 
ibility, which 1s an indicator of an organisations ability to succeed in an uncertain 
environment (Bock et al., 2012), depends on the professionals’ ability to learn, 
unlearn, and relearn. Hence, there is a need to focus on unpacking the individual 
learning processes in uncertainty. 

The uncertain nature of workplaces warrants that professionals take charge of 
their workplace learning activities (Billet, 2011; Fenwick, 2001) and self-regulate 
their learning (Fontana et al., 2015; Margaryan et al., 2009). There is a growing 
body of research exploring the intersection between working, learning, and uncer- 
tainty (Bohlinger et al., 2015b; Markowska & Wiklund, 2020). However, there is 
limited focus on the professionals’ perception of uncertainty. In particular, it is 
unclear how professionals conceptualise uncertainty in their workplace, the ante- 
cedents and consequences of the uncertainties they perceive, how they manage to 
learn during uncertainty, and the challenges they face in doing so. Examining these 
aspects are essential, as understanding how professionals perceive and learn in 
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uncertainty have implications for their job satisfaction and personal well-being 
(Flovik et al., 2019). 

Most existing research on uncertainty in organisational contexts has concen- 
trated on the impact of environmental uncertainty on organisational factors such 
as strategic foresight (Vecchiato & Roveda, 2011), entrepreneurial decision-making 
(McKelvie et al., 2011), or innovation (Freel, 2005). In their seminal work on 
organisational theory and organisational learning, March and Olsen (1975) high- 
lighted the importance of studying learning processes from an individual perspec- 
tive. However, there have been surprisingly few studies that were conducted in the 
field of learning in uncertainty, especially from an individual perspective. 

A problem that exists in fully understanding the concept of learning in uncer- 
tainty is the complexity associated with defining the objective and perceptual 
parameters of the environmental uncertainty. Weiss and Wittmann (2018) assert 
that in order to gain a holistic understanding of environmental uncertainty and the 
associated cognitive processes, it is essential first to establish the objective/tacit fac- 
tors that define the nature of uncertainties within the research context and then to 
examine the perceptual factors. However, the tacit factors are context-dependent 
and subject to change with time (Hertati, 2015;Vanevenhoven, 2012). The analysis 
presented in Chapter 17 is part of a larger research study that sought to examine 
the relationship between self-regulated learning (SRL) and perceived environmen- 
tal uncertainty (PEU) of finance professionals (Chaudhari, 2020). The research 
findings in Chapter 17 constitute the preliminary study undertaken to explore the 
nature of uncertainties within the finance sector and how the finance professionals 
perceived the antecedents and consequences associated with that uncertainty. 


17.1.1 Uncertainty in the finance sector 


The distinction between risk and uncertainty in finance is rarely made 
(Gigerenzer, 2018). Traditional finance literature encompasses the objective 
nature of uncertainty by focusing on the quantitative measures of risk (e.g., prob- 
ability models, stochastic programming), with a fundamental assumption of a 
linear relationship between risk and return (Jo & Sekkel, 2019). However, there 
is an emerging body of literature in behavioural finance that examines the sub- 
jective aspect of uncertainty in which decision maker’s perception of uncertainty 
is a substantial aspect of defining and understanding the processes of decision- 
making, innovation, and learning in uncertainty (Dow, 2010; Muradoglu & 
Harvey, 2012). In this study, uncertainty is conceptualised as per Milliken’s (1987) 
framework of PEU. Ashill and Jobber (2010) identified three variants within the 
two categories of objective and perceptual measures of environmental factors in 
Milliken’s (1987) framework — (1) measurement of individual environmental 
characteristics; (2) measuring the perceptions about the uncertainty; and (3) a 
composite measure. In their review of literature, they noted varied sources of 
objective environmental uncertainties. Perception of uncertainty is related to the 
objective environmental uncertainty (Weiss & Wittmann, 2018), which is subject 
to change with time and context (Hertati, 2015). 
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Before examining the learning processes in uncertainty, it was essential to 
establish the antecedents and consequences of uncertainties within the research 
context and how professionals perceived them. The present study addresses that 
gap by examining how the finance professionals perceive the environmental 
uncertainties in terms of their antecedents and consequences. In doing so, 
Chapter 17 answers the following research question: What are the antecedents 
and consequences of workplace environmental uncertainty perceived by finance 
professionals? 


17.2 Methods 
17.2.1 Settings and participants 


The “Finance sector" is a broad term encompassing varying domains. Typically, 
very few organisations operate in all these domains, and ones that do are globally 
distributed. With a view of interviewing participants from different domains of the 
finance sector, the Chartered Institute of Securities and Investments (CISI) - an 
independent body that is responsible for professional training and development of 
finance professionals was selected as the research site for conducting this study. It 
provided access to individuals who experienced a broad range of uncertainties in 
the finance sector. 

Selection of participants ensured an equal balance of practitioners and experts 
within the various sub-domains, to incorporate multiple perspectives from prac- 
tice and academia. The distinction between "practitioners" and "experts" is based 
on industry terminology rather than academic classification. As per CISI’s defini- 
tion, practitioners were professionals who are currently active in the finance 
industry. In contrast, experts were either academics who conducted research in 
the finance sector or organisational heads who possessed in-depth insights into 
the uncertainties in the global financial sector. This distinction certainly does not 
imply that experts lacked practical experience or that practitioners did not have 
domain expertise, yet there is value in making the explicit distinction between 
experts and practitioners. In mapping the professional learning journey from 
novice to expert, Boshuizen et al. (2006) assert that, "experts do not just know 
more than novices, they also have a different way of structuring their domain- 
specific knowledge", and that experts can provide a "certain (very successful) 
perspective on a particular domain" (p. 6). Hence the intention of the sampling 
strategy was to ensure as diverse perspectives as possible through maximum varia- 
tion sampling (Suri, 2011). 

Based on the criteria outlined above, nine professionals were identified, two of 
them were female, and seven were male. As finance is a typically male-dominated 
industry, this sample was representative of CISI membership. Interview participants 
were categorised based on their expertise, knowledge of the specific domain, and 
their practical knowledge of the sector. The average work experience of the experts 
was 33 years, while the practitioners was 15 years. 
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17.2.2 Semi-structured interviews 


Qualitative interviews allowed for capturing in-depth insights into professionals’ 
experiences and perceptions of uncertainty. They also provided flexibility in terms 
of time and methods of participation — face to face or online interviews thus mak- 
ing them the appropriate choice for answering the research question. All the inter- 
views were recorded on a voice recorder and transcribed, coded and analysed by 
the researchers using NVivo 12 software (Woolf & Silver, 2017). A combination of 
thematic analysis (Braun & Clarke, 2019) and qualitative content analysis (Mayring, 
2014) was employed to analyse the interview data. 


17.3 Results 
17.3.1 Common themes between experts and practitioners 
1 7.3.1.1 Antecedents to uncertainty 


As indicated in Table 17.1, the most commonly prevalent perception was that the 
lack of knowledge was the main precursor to uncertainty. This is expected given 
the definition of uncertainty itself is the state of not knowing. For example, when 
talking about the MiFID II (Markets in Financial Instruments Directive) regula- 
tions, Participant E4 said that: 


Table 17.1 Summary of findings 


Common themes between experts and practitioners 


Antecedents of uncertainty Lack of knowledge 
Fluctuations in the market 
Role of technology 
Lack of communication 


Consequences of Market Crash 
uncertainty Regulatory Changes 
Reassessment of risk models 
Perception of uncertainty Financial opportunity 


Learning opportunity 


Differences in perception of experts and practitioners 


Experts Practitioners 


Antecedents of uncertainty Lack of trust 
Disconnect with 
academic world 


Consequences of Lack of trust 
uncertainty Changes in training 
requirement 
Perception of uncertainty Uncertainty as something 


to be avoided (negative 
perception) 
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... With all regulation there is a certain amount of interpretation that a firm has 

to make. So, it is quite challenging for a firm. I think that is where the 

uncertainty comes in is in not knowing how to interpret that guidance. 
(Participant E4) 


Another theme that was persistently present in all the interviews was the uncer- 
tainty caused due to fluctuations in the market. Given that finance professionals 
have to deal with the highly volatile and ambiguous nature of the economic mar- 
kets, it was not surprising that market uncertainty was perceived as one of the 
common antecedents to uncertainty in the workplace. 

Additionally, the role of technology was also mentioned by all the participants as 
being responsible for introducing uncertainty in their workplace. From block- 
chain based services to robo-advisory platforms, professionals cited technology as a 
dominant antecedent to uncertainty in various contexts. For instance, Participant 
E2 cited wealth management advice-giving robots as bringing uncertainty for 
practitioners. There were also examples of algorithms driving financial decisions 
that cause uncertainty for professionals, as evidenced in the quote below: 


Ya, I mean the other interesting dimension is that it is increasingly becoming 
a less human problem. So, you know all about the amount of activities being 
allocated to algorithms. So, people don’t adapt perfectly to market conditions, 
we know that from experience. But what we don’t know yet, is particularly 
how will the algorithms that are taking note of this activity, how they respond 
to significant event, and how they interact with each other. 

(Participant P1) 


Similarly, eight out of the nine participants reported lack of communication to be 
an important factor in increasing the perception of uncertainty amongst profes- 
sionals. The scale of this lack of communication varied widely from a communica- 
tion gap between two nations to two or more individuals who were decision-makers. 
For example, the issue of people working in silos and therefore not always alert to 
issues external to their work environment was mentioned by one respondent 
(Participant E2). Likewise, Participant P3 shared a key issue highlighted in the 
Northern Rock crisis report was the lack of communication between the supervi- 
sors and management. 


17.3.1.2 Consequences of uncertainty 


Similar to antecedents, there were some common prevalent themes reported by 
experts and practitioners. For example, all the professionals gave an example of a 
market crash following a period of economic or political uncertainty. The most 
commonly cited among those were the UK Sterling International Monetary Fund 
crisis. Similarly, regulatory changes as a consequence of a period of uncertainty 
were cited as one of the common consequences of uncertainty by all the profes- 
sionals. In explaining how Basel I, II, and III regulations came about, Participant E2 
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mentioned that “every piece of uncertainty resulted in unexpected volatility and 
downside returns, resulted in more and more regulations". The experts’ view on 
consequences of uncertainty was mirrored by the practitioners with additional 
details about how the regulations imposed as a result of uncertainty had a trickle- 
down impact on the marketplace, as evidenced by the quote below: 


. [a period of uncertainty] subsequently led to a period of key regulation in 
financial market. And that key regulation resulted in the separation between 
alpha and beta and the leverage of both of those. 

(Participant P3) 


Almost all the professionals (eight out of nine) reported reassessment of risk models 
as a consequence of market uncertainty. This shows that uncertainty was perceived 
as a learning opportunity, where the lessons learnt from uncertain times were 
incorporated into the economic models, thus converting future uncertainties into 
risk. Since the definition of risk is to be able to assign probabilities to events, learn- 
ing from uncertainty events and feeding that information into the economic mod- 
els provided data for the future, thus converting the future uncertainties into risk. 
In relating his experiences of the 1987 market crash, Participant E5 elaborated on 
the process of how the uncertainty caused by the crash lead to reassessment of risk 
models and the impact it had on the market: 


.. lead people to reassess not only their risk models, but their interpretation of 
probabilistic estimates and returns and indeed what to do next in terms of 
asset allocation and so forth and whether any of the previous models worked. 

(Participant E5) 


17.3.1.3 Perception of uncertainty 


All the professionals perceived uncertainty as a financial opportunity. However, 
closer investigations of these examples that were cited as uncertainty being a finan- 
cial opportunity related to the professionals’ predisposition to think of uncertainty 
in terms of risk. For example, Participant E4 rationalised his viewpoint that uncer- 
tainty is a financial opportunity with the following argument: 


Traditionally uncertainty represents risk and risk has a trade-off through 
returns. But people are not focusing on returns at the moment, they are only 
fixing on the risks. So that certainly 1s new - this relationship between risk and 
return is manifesting itself in the way that it should. I mean, we got free money 
basically from Central Banks. 

(Participant E4) 


All the professionals also reported examples of uncertainty where they perceived it 
as a learning opportunity. These were mostly examples when the organisations/ 
individuals were unsuccessful in managing the uncertainty. This means that when 
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they failed to manage uncertainty, they viewed it as a learning opportunity. The 
following quote demonstrates an example given by a regulator when they were 
handling a ’full-blown financial crisis spilling over to the whole economy’. 


... These are not really Basel compliant, but this is something we felt that our 
market needed at that time, because of the specific risk that we faced. And as 
I said, it was too little too late - we were blamed when the whole thing started 
to fall apart. But we subsequently learned from this incident. 

(Participant P3) 


Although the experts and practitioners shared some common perceptions about 
antecedents, consequences, and ways in which they perceived uncertainty, they also 
had some differences which will be discussed in the following section. 


17.3.2 Differences in perception of experts and practitioners 
1 7.3.2.1 Antecedents to uncertainty 


A crucial difference between the experts and practitioners was that experts believed 
that lack of trust was an important antecedent to uncertainty, while practitioners 
believed that it was a consequence of uncertainty. Of the 35 items coded to this 
theme, 26 were from experts and only nine were from practitioners. Participant E5, 
who was a senior adviser, recounted how a period of uncertainty brought about 
the breakdown of trust between buyers of the firm and the employees: 


Trust was key and that created a big problem.... So, trust broke down and 
some bad things happened. I mean people did quite really naughty things 
about 20-30 years ago which created real uncertainty. 

(Participant E5) 


Similarly a disconnect to the academic world was reported as a precursor to 
uncertainty 15 times out of 21 by experts. 


... blame it (lack of information) on our lack of contact with academic world. 
However, exactly because the academic world knows more about the history 
and what is happening around the world, and how Lehman used to overcome 
things. They could have probably suggested earlier that they see bad trends, that 
in other countries they dealt with in a certain way. So probably we should have 
had more contact with the academic research centres. 

(Participant E4) 


1 7.3.2.2 Consequences of uncertainty 


A key difference in the way experts and practitioners perceived antecedents and 
consequences to uncertainty was that practitioners were more likely to emphasise 
the consequences of uncertainty, whereas experts dwelled on the causes and lessons 
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learnt. Of the 28 instances of statements coded to the theme of changes in training 
requirements preceding a period of uncertainty, 22 were made by the practitioners, 
whereas only six of those were from experts. Practitioners called for organisations to 
be *more proactive and delivering training and talk about the issues and help people 
understand better' (Participant P2). When asked about the consequences of 
technological uncertainty in a wealth management organisation, Participant P4 said: 


Yeah, I think some organisations are looking to create their own platforms and 
train their staff in using those going forward. That is one way to handle the 
uncertainty. 

(Participant P4) 


As previously mentioned, lack of trust was cited as an antecedent to uncertainty by 
the experts. Interestingly, practitioners were more likely to perceive it as a conse- 
quence that followed the period of uncertainty. Of the 18 statements coded to this 
theme, 13 were from practitioners whereas only five were from experts. Participant 
P3, used the analogy of a family going through crises and how the trust would 
breakdown if family members don't support each other to depict the loss of trust 
after a period of uncertainty: 


You know like in a family when you are going through a crisis, you and your 
partner support each other, not become overly safe and say - hmm, there is 
something wrong with you, I should put you somewhere else. This is the 
same - in good times maybe the regulator and the market should be on the 
very opposite ends and in bad times they should probably come together to 
discuss what are the issues. Unfortunately, there is lot of blame game going on - 
you did not tell us that there was market over-heating, you should have stopped 
us before we lent too much. 

(Participant P3) 


One of the critical differences in the perceptions of experts and practitioners was 
that loss of human resources due to uncertainty was discussed by all the practitio- 
ners but was never mentioned by any of the experts. When talking about the 
impact on human resources in the context of technological uncertainty in the 
wealth management sector, Participant P4 said: 


One of the regulations required all investment advisers to have a professional 
qualification, where historically people have been grandfathered in. And people 
who had worked for 30—40 years with no financial qualification suddenly had 
to get a financial qualification. That there was obviously uncertainty within the 
industry to how many people would leave the industry because of that. 
(Participant P4) 


17.3.2.3 Perception of uncertainty 


Although, both experts and practitioners perceived uncertainty as financial or 
learning opportunities, some professionals (four out of nine) attached negative 
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connotations to the word ‘uncertainty’, and as such viewed it as something to be 
managed or avoided. Practitioners were more likely to report negative perceptions 
towards uncertainty compared to experts. Of the ten statements coded to this 
theme, seven were from practitioners and only three from experts. In response to 
what professionals typically do when they perceive uncertainty, Participant P2 said: 


They do nothing, they become paralysed. So, you can find that people stop 
changing their jobs, get worried. That creates a world where internal rates of 
return projections are not acted on even though they are positive. That means 
that growth slows down. So that's the mess that we currently are in. 
(Participant P2) 


Even in statements made by experts, they talked about the negative impact of 
uncertainty in the context of practitioners, as evidenced in the quote below, by 
Participant E2 as they recounted the reactions of their employees after the 9/11 
crises: 


So, take for instance the 9/11 example. Even though it happened in the US, it 
was on the television screens. And everyone in the office was standing around 
and watching it on the screen, as the planes hit the towers. Now, we had 
contingencies in place, we had stop losses, and the European markets group 
but the people were just frozen looking at the screens. So, all the correct 
procedures were there but nothing happened. People literally could not 
respond. So again, the learning experience to me from that was that sometimes 
doing the calculations alone doesn't resolve the situation. 

(Participant E2) 


17.4 Discussion and moving forwards 


Chapter 17 examined the antecedents and consequences of uncertainties per- 
ceived by finance professionals and their general perceptions towards these uncer- 
tainties. A lack of communication and a disconnect with the academic world 
were reported as the key antecedents to uncertainty. This is in conformance with 
the findings from Novin et al’s., (1997) study that identified the need for more 
effective communication and interaction between educators and practitioners. 
‘Trust’ was yet another substantial theme mentioned by seven out of nine profes- 
sionals, in relation to the antecedent of uncertainty. Research within the change 
management literature has emphasised the role of communication and trust 
linked to the professionals’ perceived uncertainty during turbulent times. Typically, 
loss of trust was seen as a consequence of perceived uncertainty, whereas lack of 
communication was found to be an antecedent of perceived uncertainty (Arnaout 
& Esposito, 2018). The findings from Chapter 17 are in line with the previous 
studies with regards to the lack of communication being an antecedent of per- 
ceived uncertainty. However, it differs from the earlier findings, as a lack of trust 
was also noted as a potential antecedent to uncertainty along with being a con- 
sequence of uncertainty. 
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Another substantial contribution of Chapter 17 to the existing literature is the 
comparison between the perceptions of experts versus practitioners. This compari- 
son revealed that experts were more likely to indicate a lack of trust as an anteced- 
ent of uncertainty in contrast to practitioners who were more likely to report it as 
a consequence. Beyond the finance sector, a study carried out by Adobor (2006) in 
the pharmaceutical and biotechnology industry in the US and Canada, found that 
certain amount of uncertainty is necessary for the emergence of trust within the 
dealing parties, however beyond a particular threshold increase in uncertainty leads 
to a reduction in trust. Findings from this study conform to the findings of Adobor 
(2006) as practitioners reported the loss of trust as a likely consequence of uncer- 
tainty. This finding also adds to the literature on change management, as it high- 
lights the importance of studying the role of trust and communication from 
multiple perspectives. 

Furthermore, professionals recognised changes in training requirement that fol- 
lowed a period of uncertainty, whether it was objective environmental uncertainty 
or subjective perceived uncertainty. This means that when the cause of uncertainty 
was related to any external environmental factors such as regulatory changes or 
technological changes, changes in the training requirement was mandated by the 
government or organisations. However, when professionals perceived uncertainty, 
they responded by investing time and effort in reskilling or upskilling themselves. 
The findings were in line with Bohlinger et al.’s (20152) research examining work- 
place learning in uncertainty. They classified the challenges to learning under 
uncertainty under three levels: 


e At the micro (individual) level, the onus is on the professionals to learn to 
manage continuous change as it is a key qualification for their employability. 

e At the meso (organisational) level, the role of organisations is important in 
empowering their employees to manage uncertainty through upskilling, for- 
mal training, support for informal learning opportunities. 

e At the macro-level (socio-political) upskilling of the global workforce. 


An essential finding of this study was that finance professionals did not report high 
levels of negative perception towards uncertainty. For example, of the total state- 
ments coded in this study, only 1.14% of the codes were related to professionals 
talking about uncertainty negatively. This alludes to the possibility that finance 
professionals acknowledge and accept the fact that they work in a world of inher- 
ent uncertainty and they welcome it as a financial opportunity, or a learning 
opportunity or even something that they can manage. 

Within the themes for perception of uncertainty, professionals specifically talked 
about uncertainty as a learning opportunity. Most of the empirical studies carried 
out around uncertainty look at how professionals make decisions or how they 
innovate in uncertainty (Freel, 2005) but there is very little focus on how they 
learn in uncertainty and findings from this study indicate that there is potential for 
further research. 
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17.4.1 Implications for practice 


There is value in perceiving uncertainty as a learning opportunity. From managing 
workforce well-being to ensuring business continuity, learning in the face of uncer- 
tainty is a priority for organisations in order to survive and thrive in uncertain 
times. Thus, organisations and professionals should be looking towards novel ways 
of learning and building a repertoire of best practices in the new normal character- 
ised by uncertainty. An important implication for organisations is to exercise trans- 
parency and clearly communicate information related to uncertainty in order to 
prevent loss of trust of their employees in times of uncertainty. Findings from this 
chapter highlight the significance of understanding one’s learning behaviour dur- 
ing uncertainty, as it has implications for organisations as well as professionals. 


References 


Adobor, H. (2006). Optimal trust? Uncertainty as a determinant and limit to trust in inter- 
firm alliances. Leadership & Organization Development Journal, 27(7), 537—553. doi:10.1108/ 
01437730610692407 

Arnaout, B., & Esposito, M. (2018). The value of communication in turbulent environments: 
how SMEs manage change successfully in unstable surroundings. International Journal of 
Entrepreneurship and Small Business, 34(4), 500. doi:10.1504/1jesb.2018.10014545 

Ashill, N. J., & Jobber, D. (2010). Measuring state, effect, and response uncertainty: theoretical 
construct development and empirical validation. Journal of Management, 36(5), 1278—1308. 
doi:10.1177/0149206308329968 

Billet, S. (2011). Subjectivity, self and personal agency in learning through and for work. In 
The SAGE handbook of workplace learning (pp. 60-72). London: SAGE Publications Ltd. 

Bock,A.J., Opsahl, T., George, G., & Gann, D. M. (2012).The effects of culture and structure 
on strategic flexibility during business model innovation. Journal of Management Studies, 
49(2), 279—305. doi:10.1111/j.1467-6486.2011.01030.x 

Bohlinger, S., Haake, U., Jorgensen, C. H., Toivianinen, H., & Wallo, A. (20153). Working and 
learning in times of uncertainty. doi:10.1007/978-94-6300-244-8 

Bohlinger, S., Haake, U., Jorgensen, C. H., Toivianinen, H., & Wallo, A. (Eds.). (2015b). 
Working and learning in times of uncertainty: challenges to adult, professional and vocational educa- 
tion. doi:10.1007/978-94-6300-244-8 

Boshuizen, H., Bromme, R., & Gruber, H. (2006). Professional learning: gaps and transitions on 
the way from novice to expert. Heidelberg: Springer Science & Business Media. 

Braun, V., & Clarke, V. (2019). Reflecting on reflexive thematic analysis. Qualitative Research 
in Sport, Exercise and Health, 11(4), 589—597. doi:10.1080/2159676X.2019.1628806 

Brown, W., Bryson, A., Forth, J., Whitfield, K., & Kelly, J. (2011). The evolution of the mod- 
ern workplace. Industrial Relations Journal, 42(3), 313—315. doi:10.1111/j.1468-2338. 
2010.00603.x 

Chaudhari,V. (2020). Learning in uncertainty: examining the relationship between perceived environ- 
mental uncertainty and self-regulated learning of finance professionals, and the role of technology in 
supporting it. Milton Keynes: The Open University. 

Dow, S. (2010). The psychology of financial markets: Keynes, Minsky and emotional finance. 
The Elgar Companion to Hyman Minsky, 246—262. https://doi.org/10.32609/0042-8736- 
2010-1-99-113 


248 Vasudha Chaudhari et al. 


Downey, H. K., & Slocum, J. (1975). Uncertainty: measures, research, and sources of vaiiatioa. 
Academy of Management Foumal, 3, 562—577. 

Fenwick, T. (2001). Tides of change: new themes and questions in workplace learning. New 
Directions for Adult and Continuing Education, 200(92), 3—17. doi:10.1002/ace.36 

Flovik, L., Knardahl, S., & Christensen, J. O. (2019). The effect of organizational changes on 
the psychosocial work environment: changes in psychological and social working condi- 
tions following organizational changes. Frontiers in Psychology, 10(December).doi:10.3389/ 
fpsyg.2019.02845 

Fontana, R., Milligan, C., Littlejohn, A., & Margaryan, A. (2015). Measuring self-regulated 
learning in the workplace. International Journal of Training and Development, 19(1), 32—52. 

Freel, M. S. (2005). Perceived environmental uncertainty and innovation in small firms. Small 
Business Economics, 25(1), 49—64. doi:10.1007/s11187-005-4257-9 

Gigerenzer, G. (2018). The heuristics revolution: rethinking the role of uncertainty in 
finance. The Behavioural Finance Revolution: A New Approach to Financial Policies and 
Regulations, 115—134. doi:10.4337/9781788973069.00020 

Hertati, L. (2015). Impact of uncertainty of environment and organizational cultural on 
accounting information system management and implications for managerial perfor- 
mance. International Journal of Economics, Commerce, and Management, I1I(12), 455—468. 

Iwaniec-Thompson, G. (2022). The identity trajectories of older academics: workplace 
affordances and individual subjectivities. In B. Rienties, R. Hampel, E. Scanlon, & 
D.Whitelock (Eds.), Open world learning: research, innovation and the challenges of high-quality 
education (pp. 250—263). London: Routledge. 

Jo, S., & Sekkel, R. (2019). Macroeconomic uncertainty through the lens of professional 
forecasters. Journal of Business and Economic Statistics, 37(3), 436—446. doi:10.1080/073500 
15.2017.1356729 

March, J. G., & Olsen, J. P. (1975). The uncertainty of the past: organizational learning under 
ambiguity. European Journal of Political Research, 3(2), 147—171. https://doi.org/10.1111/j. 
1475-6765.1975.tb00521.x 

Margaryan, A., Milligan, C., Littlejohn, A., Hendrix, D., & Graeb-Koennerker, S. (2009). 
Self-regulated learning and knowledge sharing in the workplace. Organizational Learning, 
Knowledge and Capabilities Conference, 51(6), 51-2973. doi:10.5860/choice.51-2973 

Markowska, M., & Wiklund, J. (2020). Entrepreneurial learning under uncertainty: explor- 
ing the role of self-efficacy and perceived complexity. Entrepreneurship and Regional 
Development, 1—23. doi:10.1080/08985626.2020.1713222 

Mayring, P. (2014). Qualitative content analysis. Theoretical foundation, basic procedures 
and software solution. In Sage. http://sgo.sagepub.com/content/4/1/2158244014522633. 
short 

McKelvie, A., Haynie, J. M., & Gustavsson, V. (2011). Unpacking the uncertainty construct: 
implications for entrepreneurial action. Journal of Business Venturing, 26(3), 273-292. 
doi:10.1016/j.jbusvent.2009.10.004 

Michel, A., & Wortham, S. (2009). Bullish on uncertainty: how organizational cultures transform 
participants. Cambridge: Cambridge University Press. 

Milliken, E J. (1987). Three types of perceived uncertainty about the environment: state, 
effect, and response uncertainty. Academy of Management Review, 12(1), 133—143. https:// 
doi.org/10.5465/AMR..1987.4306502 

Muradoglu, G., & Harvey, N. (2012). Introduction/guest editorial: behavioural finance: The 
role of psychological factors in financial decisions. Review of Behavioral Finance, 4(2), 
68-80. doi:10.1108/19405971211284862 


Antecedents and consequences of uncertainties 249 


Suri, H. (2011). Purposeful sampling in qualitative research synthesis. Qualitative Research 
Journal, 11(2), 63—75. https://search.proquest.com/docview/920894910/41BC7100662 
04835PQ/1?accountid- 13042 

Vanevenhoven, J. (2012). Taxonomies of environmental uncertainty sources perceived by executives in 
the United states, Taiwan, and Mexico (Vol. 130, Issue 2). doi:10.1016/j.jac1.2012.05.050 

Vecchiato, R., & Roveda, C. (2011). Uncertainty, foresight, and strategic decision making- 
evidence from leading companies. In Forth international seville conference on future-oriented 
technology analysis (FTA) FTA and grand societal challenges — Shaping and driving structural and 
systemic transformations SEVILLE, 12-13 MAY 2011. 

Wang, X., Qi, Y., & Zhao, Y. (2019). Individual unlearning, organizational unlearning and 
strategic flexibility: The down-up change perspective. Baltic Journal of Management, 14(1), 
2-18. doi:10.1108/BJM-10-2017-0324 

Weiss, M., & Wittmann, C. (2018). Objective environmental conditions and perceived envi- 
ronmental uncertainty: cognitive models as explanation for a perceptual gap. Journal of 
Accounting and Organizational Change, 14(1), 33—60. doi:10.1108/JAOC-11-2016-0079 

Woolf, N. H., & Silver, C. (2017). Qualitative analysis using NVivo: the five-level QDA® method. 
London: Routledge. 

Zhao, Y., & Wang, X. (2020). Organisational unlearning, relearning and strategic flexibility: 
from the perspective of updating routines and knowledge. Technology Analysis and Strategic 
Management, 1—13. doi:10.1080/09537325.2020.1758656 


Chapter 18 


The identity trajectories of older 
academics 


Workplace affordances and individual 
subjectivities 


Gosia Iwaniec-Thompson 


18.1 Introduction 


With the rapid development of new knowledge, open digital networked 
technologies, and ever-evolving teaching methods and research trends in the Higher 
Education (HE) sector, older academics are as likely as other academics to continue 
developing their expertise and practice. Future projections of an ageing society and 
recent trends in delayed retirement indicate a growing number of older academics. 
Yet, empirical research considering older academics’ identity development is scarce 
and thus, there is a need to examine the changing landscape of academic practice by 
focusing on older academics’ identity development. Older academics are under- 
stood as 50 years old and over as aligned with the wider literature (Larkin & 
Neumann, 2009; ONS, 2006). 

Academic identity in Chapter 18 is studied from a sociocultural perspective, 
whereby understanding changes in academic identity focus on participation in 
practice (Wenger, 1998; Wenger-Trayner et al., 2014). Academics have multiple 
sub-identities emerging through an engagement within a landscape of practice 
(McLean, 2012). Wenger’s term “landscape of practice” is adopted here to refer to 
different communities of practice (CoP) of academics involving disciplinary, 
institutional and HE contexts that constitute academic sub-identities, for instance, 
a teacher, a researcher, a leader, or a manager. This research brings a unique 
perspective on academic’s identity development by extending Lave and Wenger’s 
work on identification by adding a dimension of identity development which 
involves an understanding of subjective interpretation of practice (Billett, 2004). In 
consequence, Chapter 18 explores tensions between different sub-identities, and 
affordances (enablers and constraints) of older academics’ identity development in 
the changing landscape of their practice. 


18.2 Ageing academics 


HE institutions are positioned at the forefront of an ageing workforce boom due 
to trends in an ageing society, the lifting of retirement age, and thus prolonged 
retirement. Kaskie (2017) suggested that the HE academic sector employs a 
greater proportion of workers over 65 relative to the general labour force, thus 
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surpassing all other sectors. In fact, the annual UK Universities report (Universities 
UK & Higher Education Statistics Agency, 2014) compared statistics from 2006 to 
2016 and found a general increase in numbers across all ages in full time academic 
roles (39.9% increase in the 31—35 year old age group, 36—40-year-olds accounting 
for a further 17.2%, and around 35% 51—55-years old). However, the proportion 
of part-time academic staff (aged 61—65 years old) increased by 44.196, while 
those in the 66 and over age group rose by a staggering 148.5% over ten years. 
Thus, by 2016, more ageing academics were remaining in part-time employment. 
Figure 18.1 illustrates these changes for full time and part-time employment of 
academic staff. 


18.2.1 Delayed retirement 


In tandem with a growing number of ageing academics, empirical research suggests 
that older academics' transition to retirement is and will be increasingly extended 
(Cahill et al., 2018; Kaskie, 2017). Currently, in the UK, the State Pension Age is 
66. At the same time, studies show that many countries abolished a mandatory age 
of retirement (Unwin et al., 2015). The UK introduced such policies in 2006 
under the Equality Act, which makes it illegal for employers to require retirement 
to employees who are at the retirement age. Furthermore, there are several studies 
exploring motivational and other factors leading academics to delay their retire- 
ment. For instance, Cahill et al. (2018) concluded that amongst many predictive 
factors to remain in employment and delay retirement, the two most important 
reasons were financial necessity and job satisfaction. Dorfman (2009) found that 
professors remained employed as they enjoyed research, whilst Winston and Barnes 
(2007) concluded that academics work beyond retirement as they are often free of 
the responsibilities they dislike, for instance, teaching and administration or man- 
agement and research (Williamson, Cook, Salmeron, & Burton, 2010). Also, Cahil 
et al. (2018) highlighted that many academics fear losing contact with their stu- 
dents and their colleagues, so they choose to stay in employment for as long as they 
can. The factor of flexibility related to working practice, working hours, and more 
opportunities of part-time work, was also identified by Koopman-Boyden and 
Macdonald (2003). They proposed that the very nature of academic work allows 
older academics to continue to undertake project-based work or consultancy, 
while moving into full retirement. 


18.2.2 Identity tensions 


Given the growing trend of ageing academics, there is a need to focus research 
on older academics. However, as the literature on older academics is limited, 
Chapter 18 drew on the broader literature about all academic age ranges when 
considering the changing nature of their practice in a dynamic HE sector, 
impacting their identity development. Contemporary research indicates that 
rapid changes in the UK HE sector and the dynamic nature of academic practice 
affect academic identity. 
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Figure 18.1 Change in age profile of full-time and part-time academic staff in UK universities between 2006-2007 and 2015-2016. 
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For example, a case study of two mid-career academics in a Cypriot university, 
discussed increased managerialism, marketisation and massification leading 
academics to negotiate the boundaries of their practice and the cost associated 
with their lateral moves (James & Lokhtina, 2018). Findings illustrated the unequal 
power relationships in the culture of academic practice that had an impact on aca- 
demic identity and engagement in practice. Also, academics’ practice and identities 
were often in constant tension, because of the struggles they encountered in their 
engagement with dynamic practice and the difficult relationships between col- 
leagues. Furthermore, a phenomenological study of identity work that academics 
engaged in as they adopted web-based tools (Bennett, 2017).The study argued that 
some academics experienced feelings of anxiety and vulnerability as they made 
changes to their practices. These changes required undertaking identity work to 
manage increased exposure through social media and negotiate the risk of failure 
associated with trying something new. This study illustrated academics were pri- 
marily motivated to be a “good teacher" rather than responding to top down pres- 
sures to bring about change and overcome these feelings. These above studies 
highlight the changing landscape of the HE sector combined with dynamic aca- 
demic practice that impacts evolving academic identity. 


18.3 Introducing study design 


The literature and initial reflections reported in Chapter 18 is part of a broader 
PhD thesis. The epistemological positioning of this study is one of constructivism. 
Constructivism seek understanding of the world in which individuals live, work 
and where knowledge is constructed by individuals in interaction with others, in 
consequence the ontological rejection of the notion ofan objective external reality 
independent of people (Creswell, 2014). To that end, this study investigated aca- 
demics’ learning and identity, using an ethnographic approach. The experiences of 
eleven academics at the OU were investigated. The purposive sampling included 
academics aged 50- to 70-year-old, including six males and five females. The data 
were collected over a six-month period. Each participant was interviewed three 
times over the course of three months either face to face or via Skype. In between 
the interviews, participants were observed on at least two occasions as they went 
about their work (for instance, during various meetings, during their desk work, 
online meetings). 

The goal of this research was for the researcher to report different perspectives 
as themes develop in the findings by relying on quotes as evidence and observation 
notes from the field. In consequence, this epistemological framing had a particular 
affinity with constructivism or interpretivist (Creswell, 2014). The focus on lived 
experiences and social behaviours of an identifiable group of people studied in 
their natural context to develop an overall cultural interpretation is defined as an 
ethnographic approach (Creswell, 2014). In making sense of how academics engage 
with those changes in their practice, the next step therefore is to explore the nature 
of academic identity. 
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18.4 Towards the conceptualisation of academic identity 


The term “identity” is a contested term due to various epistemological traditions 
that can inform the conceptualisation of the term. For instance, identity can be 
understood as one’s identification with a landscape of practice where an individual 
constitutes their identity through belonging to varied communities (Kreber, 2010). 
In contrast, identity can also be understood as a self-concept, a view which links 
identity to humanist notions of individuation, self-actualisation and gaining greater 
self-awareness of who they are (Hyland & Tse, 2012). Studies of academic identity 
recognise both individual and broader structural aspects of identity (Clegg, 2008; 
Henkel, 2005). Henkel (2000, p. 251) viewed academic identity as both distinc- 
tively “individual and embedded in the communities of primary importance to 
them”, whereas Clegg (2008, p. 329) argued that identity is a 


multiple and shifting term [which] exists alongside other aspects of how peo- 
ple understand their personhood and ways of being in the world, is not a fixed 
property, but part of the lived complexity of a person’s project and their ways 
of being in those sites which are constituted as being part of the academic. 


Both examples, without being explicit, refer to the notion of practice as a way 
of contextualising an individual in the world. Thus, both authors suggested that 
academic identity is constructed and negotiated in social interaction in the every- 
day landscape of practices.To explore the identities of academics, there is a need to 
examine their landscape of practice, which impacts the development and renego- 
tiation of their identities. To this end, the conceptualisation of identity used in 
Chapter 18 is positioned within a sociocultural and situated perspective that rec- 
ognises identity as a relational phenomenon, mediated, developed and re-negotiated 
through practice in cultural and social settings (Lankveld et al., 2017; van Winkel et 
al., 2018; Wenger-Trayner et al., 2014). 


18.5 Studies on academic identity through the CoP lens 


Several studies have explored situated and sociocultural perspectives, and in 
particular, used the CoP framework in understanding academics’ identity. For 
example, Swennen et al. (2010, p. 131) considered university teacher educators’ 
identity as “socially constructed self”, developed in participation with others in the 
landscape of practice. Whereas, Warhurst (2008) examined novice academics by 
emphasising that learning is a process of identity formation or re-formation in 
relation to the community sustaining particular forms of practice. Furthermore, 
Cahill et al. (2018) have argued for participation within a CoP as a “source of 
identity” within academic context. 

Only one study was found considering older academics’ identity formation, 
who experience transitions across the landscape of practice and changing forms of 
academic membership, drawing on Wenger's concept of CoP. Namely, James (2007) 
explored the tensions between academics different identities, and trajectories of 
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identification, that constitute the participation of academics in a landscape of prac- 
tice. James (2007) argued that academics roles, as old-timers within CoP, may be 
shaped by institutional and societal contexts over which they have little or no 
control. Therefore, in line with studies using CoP framework, the orientation in 
Chapter 18 and the study context also considers academics’ learning whereby 
engaging in social practice and negotiating meaning in the landscape of practice 
brings about changes in an individual’s identity (Lave & Wenger, 1991). 


18.5.1 Participation and reification in the landscape of 
practice 


In order to apply the lens of CoP to older academic’s practice, there is a need to 
discuss its main assumptions to understand the link between identity development 
and practice. Wenger (1998, p. 15) defined practice as “the body of knowledge, 
methods, tools, stories, cases, documents, which members share and develop 
together”. He described a CoP as having the properties of mutual engagement, 
joint enterprise, and shared repertoire. Wenger (1998) contended that individuals’ 
engagement in a CoP entails a process of negotiation of meaning which takes place 
in the convergence of two processes. Individuals move through both participation 
and reification on a trajectory from newcomers to a full participant through 
Legitimate Peripheral Participation (LPP). 

Participation in a CoP considers acting and interacting in the community of 
others engaged in the same area of practice. The participation is both personal and 
social and based on mutuality, but not necessarily equal involvement. Participation 
involves academics engaging directly in activities, conversations, reflections, and 
other forms of personal participation in academic practice. For example, academics 
teaching or working collaboratively on a research project as noted in my study. 

Reification, on the other hand, encompasses both a process and its resulting form. 
Reification involves academics producing artefacts (such as tools, words, symbols, 
rules, documents, concepts, theories, and so on) around which the negotiation of 
meaning is organised. For example, academics producing course contents, a policy 
document, or a theory in their discipline as found in the study. Participation and 
reification are complementary processes which are situated in practice. 


18.5.2 Identity and practice 


Wenger (1998) described a profound parallel between identity and practice. 
The formation of a CoP does not only involve the negotiation of meanings, 
but also, and of equal importance, the negotiation of identities. Indeed, each of 
the key concepts critical to negotiation of meaning in practice has parallel 
concepts in negotiations of identity; community as membership, shared histories 
of learning as learning trajectories, boundary and landscape as nexus of multi- 
membership, negotiation of meaning (in terms of participation and reification) 
as negotiated experience of self (in terms of participation and reification) 
(Wenger, 1999, p. 150). 
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A full CoP participant does not mean an older participant, but one that have 
developed expertise in the community. Full participation within a CoP is desirable 
as it provides access to certain knowledge, ways of knowing, and modes of conduct, 
which are inherent in the practice of a full participant (Wenger, 1999). Thus, par- 
ticipants wanting to achieve centripetal movement within the community negoti- 
ate participation and make sense of both the practice and their position in it. 


18.5.3 Multiple identities of older academics 


The practice of academics is ever changing and socially legitimated, where aca- 
demics interpret themselves as a certain individual within a certain context (Winter, 
2009). Academics’ participation in various practices results in multiple identities. 
Traditionally, universities have been institutions of both learning and scholarly 
inquiry. Thus, in general academics would engage with teaching and research. 
More recently, academics also engage in management, administration, and leader- 
ship responsibilities (Winter, 2009). A study by Deem and Johnson (2001) high- 
lighted that some academics manage to retain a balanced teaching and research 
portfolio, whilst others focus on one aspect of practice or the other. Furthermore, 
other studies showed that academics become full-time manager academics or take 
on leadership responsibilities, and thus have little or no time for teaching and 
research (Boyd & Smith, 2016). Both findings were confirmed in this study, 
whereby some academics’ identity development focused on research, teaching, or 
leadership or a combination of both or even all three areas of practice, for instance, 
research and leadership. Moreover, initial findings in this study consisted of aca- 
demics’ identities including a teacher, researcher, scholar, leader, manager, cross 
disciplinary brokerer, consultant, discipline expert. 


18.5.4 The dynamic nature of identity development in a 
landscape of practice 


Along with my study initial findings, the literature (Lankveld et al., 2017) indicates 
that institutional factors, such as introducing new initiatives, goals, directions, and 
policies at their institution, may especially influence academics! administration, 
managerial, or leadership responsibilities and hence practices. In fact, the latter is 
prevalent amongst older academics who often take on management and leadership 
roles based on their long experience in academia. These roles are often temporary. 
McAlpine (2012) concluded that academics conceptualise themselves as having 
multiple identities due to the changing forms of academic community member- 
ship. Thus, they can become leaders and managers alongside their teaching identity. 
Furthermore, the boundaries of HE are increasingly “porous” and academics are 
developing a wider ranges of identities influenced by activity beyond the university 
(Clegg, 2008), such as international projects, cross-disciplinary research or govern- 
mental and private professional bodies and employers (Martin, Lord, & Warren- 
Smith, 2020). This leads academics to be involved in the landscape of CoP which 
extend beyond their immediate CoP. 
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Development of all these identities at the same time may not always be possible, 
and some may be more pronounced than others, or they may clash altogether. 
Winter (2009) suggested contradictions and conflicts that arise from these compet- 
ing identities as academics enact their multifaceted practice. This leads academics 
to experience competing priorities in their identities (Bolden, Petrov, & Gosling, 
2009). Furthermore, some authors have argued that academics most strongly iden- 
tify with their discipline rather than their workplace (cf. Henkel, 2005) as con- 
firmed in the findings of this study. For example, one academic in particular talked 
about his disciplinary work outside of his workplace with a great passion. Indeed, 
practices change and evolve over time and over contexts and new challenges 
require new ways of practising which affect identities of the academics. 


18.5.5 Modes of identification 


By participating in the landscape of practice academics engage in learning activities 
that imply changes in individuals! identities (Lave & Wenger, 1991). Wenger empha- 
sized that these changes involve engagement, imagination and aligning one’s skills, 
knowledge and identity with the community, thus learning arises out of engagement 
in the practice of the CoP rather than merely engaging in educational activity (Lave 
& Wenger, 1991). Wenger suggests three modes of identification that positions an 
individual in the landscape of CoP: engagement, imagination, and alignment. 


18.5.5.] Engagement 


Engagement, being the most immediate relation to practice, involves engaging in 
work activities, working collaboratively or alone. Engagement provides direct 
experience of what the community is all about, whether this experience is one of 
competence or incompetence and whether participants develop an identity of 
participation or non-participation. For example, the findings indicated that some 
academics focused their engagement on the development of their research identi- 
ties, but disengaged with leadership or teaching practices, and indicated no desire 
to get competence in these practices. Conversely, other academics practice focused 
on developing leadership identities which left limited time for research. 


18.5.5.2 Alignment 


Alignment involves a two-way process of coordinating perspectives, inter- 
pretations, actions, and contexts of a participant's trajectory into the CoP. For 
example, a teaching academic aligned some of their teaching methods to those 
practised in their department but also introduced innovation by suggesting using 
Twitter in their course, which changed aspects of the teaching culture of the 
department. Alignment becomes very deep aspects of participants identities as it 
concerns power which can amplify but also disempowers sense of what is 
possible. For example, in my study one academic struggled to establish his area 
of work as “proper”. 
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18.5.5.3 Imagination 


Imagination involves an understanding of how we belong or not to a CoD it is 
extrapolating our own experiences through time and space. For example, the study 
found that research-engaged academics imagine belonging to a wide disciplinary 
community even though they may be the only ones doing research in a certain 
area of their workplace. Indeed, the study findings showed that all participants 
expressed ongoing interests in research and scholarship practice and its value in 
their disciplines and workplace. Yet, some participants admitted having no time to 
engage in the activity itself. This finding has implications for participants! identity. 
Namely, participants recognise the importance of research in the identity of an 
academic and may even align their identity to the place of research even if they do 
not engage in primary research themselves. 


18.6 Subjectivity in identity development 


Thus far, Chapter 18 has argued that the development of academics’ multiple 
identities is both social and CoPs dependent. However, researchers urge that both 
the context and the way that individuals make sense of this context should be 
examined when investigating professional identity. It is at this point, where 
Wenger’s work has been critiqued, as he provides limited attention to individual 
subjectivities to understand the choices individual's make in participation in prac- 
tice (Fuller et al., 2005). 

Each academic s experience will be unique due to "the inevitable negotiation 
between the workplace norms and practices and the individuals’ subjectivities and 
identities" (Billett 2004, p. 114). Individuals enter workplaces with their own idio- 
syncratic personal learning experiences, perceptions and memories which will 
shape their dispositions to particular participatory practices For this reason, research 
focusing on academic identity should consider the subjectivities that individuals 
employ within their academic landscape of practice. Therefore, the processes of 
alignment (congruence and incongruence) with the values, practice and norms of 
the community and also individuals’ biographies, norms and values are seen as 
integral to their academic identity formation. This duality is conceptualised by 
Billets (2004) notion of co-participation, and describes how access to identity 
development is afforded, on the one hand, and how workers elect to engage with 
what is afforded to them which shapes their identity, on the other. Each individu- 
al's understanding is a subjective reconstruction or construal of practice that shapes 
one’s identity (Hodkinson et al., 2008). 


18.7 Affordances for identity development 


By focusing on the situated nature of practice, Eraut (2009) pointed out that the 
different local goals, norms and role boundaries of workplaces create varied 
affordances to engage in participatory opportunities. Research suggests that the 
myriad forms of academic identities are constituted by multifaceted contexts. 
These are: the academic’s workplace environment, the wider context of national 
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and international HE, their discipline context, as well as social interaction with 
peers and students.These contexts can play an important role in either strengthening 
or inhibiting the development of academic identity. Furthermore, each of these 
contexts can have a varying impact on academic identity (Lankveld et al., 2017). 
Furthermore, these contexts can be understood as learning affordances (Billett, 
2004) which can constrain or resource the identity development. These highlight 
potential, disempowering, or exclusionary experiences in which affordances may 
not be evenly available. 


18.7.1 Affordances enhancing identity development 


One of the affordances that has been mentioned by several studies that align with 
the findings of Chapter 18 was a collegial and supportive environment in the aca- 
demics workplace and across discipline networks (Martin et al., 2020). Feeling sup- 
ported and respected by their colleagues meant for academics that they flourished 
and developed their identities by giving each other emotional and practical support 
(Lankveld et al., 2017). Furthermore, the study found being a recognised expert in 
a discipline afforded idenüty development by being invited to conferences or being 
approached by colleagues to participate in collaboration, thus developing expertise 
identity in respective discipline. The participants’ strong identification with research 
and scholarship translated into the focus on intensive applied research that stressed 
collaboration and practical implications of their research across disciplines. The 
expressions of commitment, pride, and passion were evident in academics’ accounts, 
reinforcing their identity as researchers and scholars. 

The participants in this study also interpreted their past experiences as an 
affordance for identity development, for example, their teaching experience. 
Through reflecting on these experiences, they were able to develop themselves as 
professionals. These findings align with the conceptualisation of identity as negoti- 
ated in everyday interaction and experience (Archer, 2008). 


18.7.2 Affordances restricting identity development 


In contrast, other studies reported that the work environment had a constraining 
impact on identity when academics perceived their departments to be competitive, 
hierarchical, lacking in trust, or valuing research more than teaching. In such 
environments, academics felt isolated and inhibited to ask for help and support to 
develop either their teaching or research identities (Cahill et al., 2018). An exam- 
ple from this study's findings showed that some academics felt that their career 
progression was restricted, as well as having limited formal developmental 
opportunities. 

The findings of this study also indicate that although the diverse prior experi- 
ence in participants’ practice meant that they could be involved in different prac- 
tices, it also created some tensions. Thus, some of academics’ identities were an 
obstacle for the development of other academic identities. Indeed, several academics 
discussed struggles with keeping up with research practice due to management and 
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leadership responsibilities and the specific nature of their discipline. These findings 
are aligned with other studies mentioned by Bolden et al. (2009). 


18.7.3 Identity development in Open World Learning 


With the rapid development of open world learning technologies older academics 
continue developing identities. However, the challenge of encouraging older aca- 
demics in HE to adopt to these changes is frequently noted within the academic 
development community. The initial findings of this study noted two aspects of 
using social media. On the one hand, the study that the benefits of utilizing social 
networking when carrying out research were clearly recognised by many partici- 
pants. Consistent results across studies have revealed that social media sites provided 
a convenient environment for scholarly communication and research dissemina- 
tion. These findings aligned with other studies (Bennett, 2017; Kara et al., 2020). 
Furthermore, the findings reported that academics used ResearchGate and 
Academia.edu for keeping themselves up to date in their field, promoting their 
work online, and maintaining their professional image. 

On the other hand, there were some barriers hindering the adoption of social 
media. The challenge of encouraging academics in HE to adopt social media to 
support their teaching, research, and management roles was frequently noted 
within the academic development community (Brown, 2013), especially amongst 
older academics (Gelade, 2013). For example, studies considering identity develop- 
ment uncertainties in adopting social media in teaching and research practice 
(Bennett, 2017; Kara et al., 2020). Indeed, Greenhow and Gleason (2014) found 
that age played an important role in the decision to adopt social media for teaching. 
The findings of this study aligned with Bennet’s study (2017) whereby some aca- 
demics discussed uncertainties related to the use of social media, yet these were 
alleviated by time they spent on getting used to new practice for the benefit of 
their research and teaching. 


18.8 Discussion and moving forwards 


The aim of Chapter 18 was to problematise the dynamic nature of older academics’ 
identities and report some initial findings. The scarce literature on older academics 
necessitated drawing on wider literature. This calls for more research focusing on 
older academics given delayed retirement and the increased ageing academic pop- 
ulation. The conclusions of James's (2007) study compared to the findings of this 
study revealed that an individual subjectivity (Billett, 2004) in understanding older 
academics identity development was not addressed. Thus, this study builds on pre- 
vious work by adding an additional framework in order to understand identity 
development of older academics. 

Chapter 18 indicated that older academic identity consists of sub-identities 
across a landscape of practice, including but not limited to: teacher, researcher, 
leader, consultant, discipline experts. The findings indicated that the academics 
were recognised for their expertise and thus seen as sources of knowledge. However, 
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multiplicity of academic identities means their practice is contested, hence future 
research should address the ways how those tensions could be addressed. 
Participating in the landscape of practice, academics involve dynamic context of 
HE sector and evolving workplaces including technological changes. The chapter 
indicated that both affordances (enablers and obstacles) of the practice but also 
individual subjectivities shape academics identities development. More research is 
necessary to understand how workplaces could use these insights to foster an envi- 
ronment where particularly the barriers are addressed. For example, the use of 
open access and social media use was discussed, both as an affordance that helped 
academics to promote their work, but also as an uncertainty that has implications 
for more focused help for individual academics’ development. 


18.8.1 Implications for practice 


The practical implications of distinguishing multiple identities of older academics’ 
could enhance an individual's understanding for thinking about their own profes- 
sional development. There are several tensions caused by the dynamic and multiple 
identities of older academics. This could both inform practice, concerning, for 
example, emotional processes involved in these changes, but also encourage 
younger and older academics to negotiate time to maintain the identities that were 
pushed on the periphery for maintaining an appropriate balance. 
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Chapter 19 


Reflecting on the main findings 
and practical applications 


Bart Rienties, Regine Hampel, Eileen Scanlon and 
Denise Whitelock 


19.1 Introduction 


The main objective of this book Open World Learning: Research, Innovation and the 
Challenges of High- Quality Education was to establish an informed theoretical and 
methodological basis for research and practical application of open world learning. 
As highlighted throughout this book, open world learning gives unprecedented 
access to information, knowledge, and education and provides support to learners 
across the globe. However, it is not the educational technologies themselves that 
represent the biggest change, but the opportunities for openness that flow from 
their thoughtful application, in the form of availability of and access to open and 
"closed" learning opportunities (Ferguson, Jones, & Scanlon, 2019; Hampel, 2019; 
Nguyen, Rienties, & Whitelock, 2022; Rizvi, Rienties, Kizilcec, & Rogaten, 2022). 

The main question of this book was: How can open world learning supported 
by technology help and/or hinder tackling the global challenges that open 
and high-quality education faces? First, this book provided several integrated 
and cohesive perspectives of the affordances and limitations of open world learning. 
The eighteen chapters brought together a range of research communities, including 
colleagues in artificial intelligence (Hillaire, Rienties, Fenton-O’Creevy, Zdrahal, & 
Tempelaar, 2022), computing (Iniesto & Hillaire, 2022), education (Anastasiou, 2022; 
Mohamud, Buckler, Pitt, & Twining, 2022; Srisontisuk, 2022), educational psychol- 
ogy (Hall, Herodotou, & Iacovides, 2022), human-computer interaction (Iniesto, 
McAndrew, Minocha, & Coughlan, 2022), language education (Conde Gafaro, 2022; 
Vogiatzis, Charitonos, Giaxoglou, & Lewis, 2022), learning analytics (Korir, Slade, 
Holmes, & Raenties, 2022; Nguyen et al., 2022; Rizvi et al., 2022), linguistics (Chua, 
2022; Rets, Stickler, Coughlan, & Astruc, 2022), and professional learning (Chaudhari, 
Littlejohn, & Cross, 2022; Iwaniec-Thompson, 2022). 

Second, this book featured a wide range of open world learning topics, ranging 
from theoretical and methodological discussions to empirical demonstrations of 
how open world learning can be effectively implemented, evaluated, and used to 
inform theory and practice. This book provided in-depth analyses of the (potential) 
benefits and limitations of open world learning by bringing together insights from 
387,134 learners and educators learning and working in 136 unique learning 
contexts across the globe (e.g., blended language learning in Italy and the UK, 
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Internet kiosks in Uganda, MOOCs involving learners across the globe, online 
courses in the UK, science learning in Greece). This interconnected body of work 
not only has tremendous reach, but by comparing and contrasting different learning 
experiences of academics, children, educators, gamers, learners, instructional 
designers, managers, professionals, students, teachers, and young adults in a range of 
local, national and international settings, the sum of these 136 learning contexts is 
more than each of its individual parts. 

Third, this book brought together a range of innovative uses of technology in 
open world learning, such as digital games (Hall et al., 2022), Futurelearn (Chua, 
2022; Conde Gafaro, 2022; Iniesto et al., 2022; Rizvi et al., 2022), iPads (Srisontisuk, 
2022), open educational resources (Rets et al., 2022), solar-powered Internet 
kiosks (Mohamud et al., 2022), (student sourced) sentiment analysis classifiers of 
online chat (Hillaire et al., 2022), WhatsApp (Vogiatzis et al., 2022), and Youtube 
(Anastasiou, 2022). A range of complex and interlinked research methodologies 
have been used to understand how educational technology is used by people in 
open world learning settings. Several innovative methodologies were used, includ- 
ing quantitative methodologies such as discourse analysis (Chua, 2022), eye- 
tracking (Rets et al., 2022), learning analytics (Nguyen et al., 2022; Rizvi et al., 
2022), machine learning (Hillaire et al., 2022), and (psychometric) online surveys 
(Conde Gafaro, 2022; Hall et al., 2022; Korir et al., 2022). Furthermore, a range of 
in-depth qualitative methodologies were employed throughout this book, includ- 
ing (ethnographic, longitudinal) observations (Anastasiou, 2022; Iwaniec- 
Thompson, 2022; Mohamud et al., 2022; Srisontisuk, 2022), and thematic analysis 
of artefacts, documents, interviews, and other qualitative data (Anastasiou, 2022; 
Iniesto et al., 2022; Mohamud et al., 2022; Srisontisuk, 2022;Vogiatzis et al., 2022). 
Finally, several chapters specifically embraced mixed method approaches (Iniesto 
et al., 2022; Rets et al., 2022; Rizvi et al., 2022). 

In the remainder of this chapter, we will not try to summarise each of the chap- 
ters, but rather try to place the findings into a wider context of the open world 
learning framework previously described in Chapter 2 (Rienties, 2022). This 
framework helps to depict what we have learned about what works in open world 
learning, what practical advice we give to learners, educators, and institutions, and 
finally what the next steps in open world learning research could be. As is evident 
throughout this book the enablers and disablers for open world learning continu- 
ously change the fluidity of the open world learning framework. This means that 
the size and space of open world learning experienced by individual learners and 
organisations in any specific context might be substantially different from other 
learners and organisations, and will inevitably change over time as technology, 
society, and people continuously interact in a flux. 


19.2 What is now known about what works in Open 
World Learning 


As argued in Chapter 2, while substantial progress had been made in mapping and 
reviewing the macro (ie., regional, national, international, global) trends and 
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concepts of open world learning until 2014, there was a paucity of evidence-based 
research how learners and educators make use of open world learning on a meso 
(i.e., institutional, cross-institutional, cross-discipline) and micro level (within insti- 
tution, module, student). Chapters 14 and 15 specifically focussed on a meso level 
how MOOC organisations (Iniesto & Hillaire, 2022) and course designers at the 
Open University (INguyen et al., 2022) make decisions in terms of how to design 
effective and inclusive learning designs and experiences. 

Six chapters (Chaudhari et al., 2022; Iniesto et al., 2022; Iwaniec- Thompson, 
2022; Mohamud et al., 2022; Rizvi et al., 2022; Srisontisuk, 2022) combined a 
meso perspective with more fine-grained micro-level analyses to understand how 
individual learners and educators are making decisions within a broader context. 
For example, in Chapter 17 (Chaudhari et al., 2022) a convincing narrative is pro- 
vided how experts and practitioners in the finance sector dealt with uncertainty, 
and how this was influenced by both individual as well as meso-factors. Similarly, 
in Chapter 8 Rizvi et al. (2022) showed that how learners from across the globe 
engage with MOOCs is influenced both by micro factors (e.g., demographics, 
prior knowledge, motivation) as well as meso or even macro factors (e.g., culture, 
national educational provision). 

Equally important, seven chapters (Anastasiou, 2022; Chua, 2022; Conde Gafaro, 
2022; Hall et al., 2022; Hillaire et al., 2022; Korir et al., 2022; Rets et al., 2022) 
specifically focussed on the lived experiences of learners and educators engaging 
with open world learning on a micro-level. For example, Chapter 11 (Korir et al., 
2022) explored how 447 students reacted to variations in privacy risks and/or 
benefits interventions in an experimental design. Chapter 3 (Anastasiou, 2022) 
explored children who received a different sequence treatment in using digital 
stories in science, with the results showing that children from an early age can 
creatively use a range of open world learning tools to tell their own stories and 
make meaning in their own language. 

Irrespective of whether the focus was on a meso or micro level, the two main 
lessons from this book are (1) technology is not neutral; (2) open world learning is 
not necessarily open for everyone. 


19.2.1 Technology is not neutral 


Throughout this book how, why and with whom learners and teachers are engag- 
ing in open world learning was substantially influenced by the affordances and 
limitations of a respective (educational) technology. For example, while WhatsApp 
is omnipresent in many peoples lives, just starting a suite of language tasks on 
WhatsApp to learn German is not a guarantee for success (Vogiatzis et al., 2022). 
While tablets provide teachers and children with opportunities to gain new insights 
and knowledge, at the same time tablets may distract from natural play and interac- 
tion (Srisontisuk, 2022). How MOOCs are designed often favours Western learn- 
ing approaches as they are designed by Western educators (Rizvi et al., 2022), thus 
"forcing" learners from other cultures to adopt an unfamiliar learning approach. In 
addition, as educators’ choices in terms of which learning design activities are 
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included or excluded in a particular week of learning are based upon their own 
experiences and expectations, this inevitably has profound implications for how 
learners are expected to study (INguyen et al., 2022). 


19.2.2 Open world learning is not necessarily open for everyone 


Building on the first main lesson, even with the best Internet access, IT infrastructure, 
and educational support available, not everyone will benefit equally from the 
opportunities of open world learning. For example, Iniesto et al. (2022) showed 
that learners with accessibility needs often struggle to make use of open world 
learning. In MOOCs interactions, for example, some types and groups of learners 
are often more present than others (Chua, 2022; Rizvi et al., 2022), thereby poten- 
tially reducing the (heard) voices from other learners. As most open world learning 
opportunities are created in the medium of English, non-native English speakers 
are often put at a disadvantage in terms of benefitting from engagement with 
OERs or MOOCs for learning (Chua, 2022; Rets et al., 2022). While the com- 
munity approach of Internet kiosks in Uganda by Mohamud et al. (2022) showed 
that learners in a community can teach each other to learn to use open world 
learning tools, this approach may not necessarily work in other contexts. 

While the authors in this book acknowledge that technology is not neutral, and 
that not everyone may benefit equally from open world learning, throughout the 
book there are markers and evidence-based findings supported by robust method- 
ologies how you as learner, educator, or as manager can ensure that you can benefit 
from the powers of open world learning. Implementing some of the practical 
advice may help you to become a more successful open world learner and/or edu- 
cator, as illustrated in section 19.3. 


19.3 Practical advice to support and mediate effective 
open world learning 


Empowering learners (how to become an effective open world learner) 
e Set your own goals what you want to achieve in open world learning 
(Chapters 5 and 9). 
e Follow the learning design schedule set in open world learning, and adjust 
where needed (Chapter 8 and 14). 
e Get involved in (co-)designing open world learning (Chapter 9 and 15). 
e Think carefully about including open and inclusive language. This will 
increase constructive engagement with other learners (Chapter 6). 
* Position yourself as a digital story maker and take the opportunity to do so 
in every step of the creation process in open world learning (Chapter 3). 
Empowering innovative technologies (how to use them) 
e There is no single universal learning design that works for all learners 
(Chapters 5, 6, 8, and 9). 
e The mere use of innovative tools like WhatsApp or Massive Open 
Online Courses (MOOCs) cannot guarantee interaction and knowledge 
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construction amongst your learners. These tools need to be appropriately 
embedded into your learning design (Chapters 4 and 6). 

As long as the linguistic accessibility of online materials is ignored, and 
these resources continue to draw on native speaker capital in language, the 
capacity of these resources to widen access to quality education will only 
remain that: a potential (Chapters 7 and 9). 

When using artificial intelligence tools to identify complex patterns in 
data (e.g., emotions, discourse), you should consider how to align the lived 
experience of students to model highly subjective topics (Chapters 6 and 13). 
Think about possible concerns regarding the usage of smartphones and 
tablets with young children (Chapter 16). 

Solar-powered Internet kiosks can support low-income communities 
to achieve their desired goals and facilitate soft skills development 
(Chapter 10). 

As a developer you could use the Creative Gaming scale as a guide on what 
aspects of creativity are most important to players and learners (Chapter 12). 


Empowering educators (what you need to do as educator) 


It is crucial for you as educator to identify how learners set and assess their 
goals in open world learning (Chapter 5). 

Simplify your language! We repeat: Simplify your language! (Chapter 7). 
There are many potential misalignments between what you as an educator 
think your learners do and what they actually do (Chapter 14). 

As educator you should seek a better understanding of your learners and 
their needs (Chapter 9). 

Set clear guidelines on specific, realistic, measurable, and attainable goals 
so that your learners can effectively self-regulate their learning in an open 
world (Chapter 5). 

It is important to have frequent check-ins with your learners, not only at 
the point of an assignment deadline but throughout their learning process 
(Chapter 14). 

Simplification strategies in writing online materials, such as splitting 
sentences, choosing words of a shorter length and higher frequency, and 
using fewer nouns and more connectives between/within sentences have a 
beneficial effect on the text processing of learners, in particular non-native 
English speakers (Chapter 7). 

Use digital stories to communicate complex information (Chapter 3). 
There is a strong need for flexible, culturally adaptive learning designs of 
open world learning, taking a balanced approach by combining different 
types of learning activities (so not just more text and videos) (Chapter 8). 


Empowering your educators (how to get the best out of your staff and 
learners) 


Any online course development processes need to be reviewed from the 
early design stages to produce accessible content (Chapters 9 and 15). 
Your design focus should change from meeting legislative requirements to 
meeting learners’ needs (Chapters 9 and 15). 
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e Extensive training of educators in using innovative technologies and 
pedagogies is needed to ensure the full potential of open world learning. 
So do provide more training and support if your educators or staff need 
them (Chapters 4 and 17). 

* Institutions should examine ways to empower students with respect to the 
use of their data by allowing them to indicate whether and which data 
items they would be willing to share (Chapter 11). 

e Be transparent regarding the use of student data (Chapters 11 and 13). 

e There are several tensions caused by the dynamic and multiple identities 
of (older) academics. Becoming familiar and proficient with open world 
learning takes time and resource (Chapter 18). 

e Organisations need to exercise transparency; if there is uncertainty, this 
needs to be clearly communicated in order to prevent loss of trust of their 
employees (Chapter 17). 


19.2 What is next for open world learning? 


As indicated in Figure 19.1, a lot of progress has been made in understanding how 
people make use of open world learning, both on a meso as well as a micro level. All 
but one Chapter specifically focussed on how learners and educators used particu- 
lar open world learning approaches in their context. Some of this usage of open 
world learning seems to depend on individual people factors, such as accessibility 
needs (Iniesto et al., 2022), age (Iwaniec- Thompson, 2022; Srisontisuk, 2022), con- 
ceptions of teaching (Nguyen et al., 2022), emotions (Hillaire et al., 2022), engage- 
ment (Chua, 2022; Rizvi et al., 2022; Vogiatzis et al., 2022), language skills (Chua, 
2022; Rets et al., 2022), self-regulation (Conde Gafaro, 2022), and socio-economic 
factors (Rizvi et al., 2022). 

Furthermore, some substantial progress has been made in terms of practices, as the 
practices people and institutions are surrounded by influence how they engage 
with open world learning. For example, Chapter 9 (Iniesto et al., 2022) showed 
how the organisational practices around MOOC providers and the legislative 
context substantially influenced how accessible a particular learning unit was. 
Chapter 16 indicated that how children were able to use iPads in the classroom 
depended in part on the practice in their respective school (Srisontisuk, 2022). 
Overall, our 136 unique learning contexts discussed in this book provide a rich and 
diverse overview of some of the practices used in open world learning. 


People Places Practices Properties 
Meso 1 0 2 2 5 
Meso/micro 6 2 6 1 15 
Micro 8 1 1 4 14 
15 3 9 7 


Figure 19.1 Heatmap of 4P themes covered in this book. 
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While in 2014 there were hardly any studies on the properties of open world 
learning technologies and data, substantial progress has been made in this book, 
with seven contributions focusing on how data use might enable or thwart open 
world learning. For example, Chapter 11 (Korir et al., 2022) showed that most of 
the 447 UK students involved in an online experiment focusing on privacy and 
learning analytics were reasonably comfortable to share their data with higher 
education institutions. Chapter 12 (Nguyen et al., 2022) showed that OU educa- 
tors made substantial use of student engagement data to determine how to design 
online courses. At the same time, with the triangulation of more and more data and 
Artificial Intelligence becoming more intertwined with education (Rienties, 
Kohler Simonsen, & Herodotou, 2020), there are substantial concerns about how 
algorithms are potentially making decisions (e.g., in- or excluding learners on a 
particular invention, profiling, automatic feedback) that could influence behaviour 
and performance of learners (Baker & Hawn, 2021). 

As evidenced by Figure 19.1, relatively few Chapters focussed on places, which 
is perhaps surprising as places where people learn, work and live might substan- 
tially impact on how they get access to open world learning opportunities. With 
governments across the globe setting national agendas for (open and closed) educa- 
tion, and some governments restricting access to Internet and knowledge, more 
research is needed how places impact on learners and learning. Beyond actively 
restricting content and access to open world learning, even when learning activi- 
ties are openly available, this does not necessarily imply that users will universally 
make sense of them in the same way. For example, in Chapter 8 Rizvi et al. (2022) 
showed that MOOCs were not culturally inclusive, with large differences in 
engagement patterns by 49,582 learners in 10 Futurelearn MOOCs. Therefore, 
there is an urgent need to understand how the construct of places influences open 
world learning and the lived experiences of people who live in those places. 

Given the growing maturity of open world learning research, now is the time to 
usefully gather, compare and contrast more data on a wide range of experiences 
from a micro-meso level to inform an evidence-based macro perspective of open 
world learning. As editors we hope that this book has been inspiring, and that you 
might try some of these tools, approaches, and pedagogies in your own context. 
Please share your perspectives and insights with us and tell us what has worked, and 
what has not worked via our social media (@IETatOU). Only together can we 
help to further empower open world learning. 
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